Laravel Bytes

Never Miss a Webhook

2024-02-26T00:00:00+00:00

Fly.io converts Docker images to fast-booting VMs and runs them globally. This is great or ingesting incoming data! Check out how to scale Laravel on Fly.io.

If your app ingests data from users (webhooks or similar), then it’s important not to miss any of that data.

This is actually a bit scary. We need, like, a lot of uptime! The typical way to increase uptime smells like “scaling” and often feels like expensive premature optimization. How do we do this!?

Here’s how I thread the needle between complex and, well, cheap.

We’ll cover a few problem areas and how to think about them:

Handling Bugs
Handling Spikey Traffic
Handling Availability

The Bug Problem

Who among us has not deployed bugs into production? What often happens is we adjust something way over here but accidentally cause errors over there. This is bad when “over there” is your ingest code.

We want to reduce our chances of adding bugs to our ingest code.

The Bug Solution

We are not about to write less bugs. Instead, we need to protect ourselves from our bugs. Here’s a few ideas.

KISS

Our ingest code should, in all cases, be simple. Ideally the ingest data we care about (or perhaps the entire HTTP request?) is stored as quickly as possible, and then a queue job is fired off to do the actual work needed.

The code path should be simple, and preferably have very little “hidden” complexity.

To that end, consider which middleware can be stripped out of the hot path of ingestion. CSRF protection is one, and perhaps even rate limiting and/or authentication can be moved out of PHP and into, like, Nginx + LUA (check out Openresty).

    # An example invokable controller for ingest
public function __invoke(Request $request)
{
    // TODO: Validate the data if needed (e.g. validate a signed request)

    // Save webhook to storage (likely s3 or similar)
    $webhook = json_encode([
        'method' => $request->method(),
        'headers' => $request->headers->all(),
        // String representation of the body, 
        // I'm assuming no files are sent.
        // An alternative to this would be base64'ing
        // a binary format of the body
        'body' => $request->getContent(), 
        'uri' => $request->getUri(),
    ]);

    $reqId = $request->header('fly-request-id') 
        ?: Str::uuid()->toString();

    $file = sprintf("%s.json", $reqId);

    // An S3 (compatible) storage
    Storage::disk('webhook')->put($file, $webhook);

    // Queue a job to process the webhook
    ProcessWebhook::dispatch($file);
}

  

Even this code relies on external services, albeit reliable ones (S3 + SQS for me). Choose your dependencies carefully!

Use a Separate Code Base

A great way to avoid bugs is to create a separate code base for ingest. This is attractive as the code base will likely be much smaller - it has a reduced surface area.

We can deploy this separately (and perhaps much more rarely) - letting it sit around in glorious, reliable boredeom for years.

The drawback of this is that you don’t have all of your business logic, models, and other supporting code hanging around. Probably. There are creative ways around that, too, but why add complexity?

This means it’s generally “harder” to do logic in the ingest application. If you don’t mind me sitting in my ivory tower for a second, I’d simply suggest not having logic in the ingest application. Preferably, any other processing can be done by the “main” application via a queue worker.

Reusing Our Code Base

If you need (or want) to re-use your code base, perhaps host it as a second, separate app (e.g. the same code base but at a subdomain like ingest.my-app.com).

This has a bunch of benefits. All of our code is there with our models, relationships, business logic, and so on. We can re-use that as-needed when ingesting data.

We can also smoke-test the code base before deploying udpates to the ingest app. Since we’re most likely going to break something after a deployment (bugs aren’t bugs until they’re shipped to production), we can update our main application first. Once we have a known stable deployment, we can deploy updates to the ingest app.

The Spikey Traffic Problem

Perhaps the most common (but often overlooked) problem is how easily spikes in traffic can overload our PHP servers!

The most common PHP setup is Nginx -> PHP-FPM -> your PHP code. In this setup, Enemy Number 1™ is request concurrency, and it’s all because of PHP-FPM’s max_children setting.

PHP-FPM spins up multiple child processes to handle requests. Each child process handles requests in series (one at a time). We only get request concurrency by spinning up more processes! However, once we reach a certain concurrency (max_children), FPM won’t create additional processes and instead returns errors.

It turns out the max_children setting is pretty low by default.

So, our job is to deal with that most-likely scenario (and others if we can).

The Spikey Traffic Solution

There’s a few solutions to spikey traffic. One is to just remove PHP-FPM and therefore it’s (artificial) limits. Another is to over-provision your server(s) so additional traffic can be absorbed.

We’ll talk about using more servers next, first let’s talk about handling spikey traffic within each app isntance.

Octane

For Laravel, you may want to use Octane for ingest.

The main benefit is ditching PHP-FPM. This way we get around the max_children setting which can artificially impose limits. You could set it to some super high value tho, but I like removing it out of the equation if I can.

The other benefit to Octane is its increased throughput - higher requests per second. This handles spikey traffic better!

If you’re running on Fly.io, you want to be sure your fly.toml concurrency settings aren’t too high. If the Fly Proxy thinks the server can handle more requests per second than it can, it can starve the VM.

That’s a good problem to have, people are actually using your app! One simple way around this is “vertical scaling” - give yourself a larger server.

    # look, a larger server!
# increase the size, and run `fly deploy`
[[vm]]
  memory = "4gb"
  cpus = 4
  cpu_kind = "performance"
  processes = ["app"]

  

The Availability Problem

Most projects start life on a single server. “We’ll scale out when we get enough users”, we declare.

Many projects live (and die) there, but some few are lucky enough to have users. Then a new fear creeps in: What if my server goes down? We’ll miss that precious user data.

Now we’re looking at adding extra servers. That means more complexity! We need a load balancer, we need to deploy to multiple servers, and we need your database/redis/whatever to live somewhere else.

How do we do all of that!?

The Availability Solution

Let’s tackle what HA means. It comes in a few flavors:

Server redundancy
Auto-scaling (in and out)
Zero-downtime Deployments

Here are some ideas!

Scale Out

We want multiple instances of our ingest application. This gives us 2 things:

Higher availability (if a host fails, or something equally wonky)
More “throughput” - more requests per second when load balancing between instances

However, once we get more servers, we need a load balancer. In these situations, I usually reach for something managed. On Fly, it’s just…kinda there for you from the beginning. Everything goes through the Fly Proxy, which is a “reverse proxy” already - a load balancer (and more).

To scale our application, we can run a few commands:

    # Scale to 10 instances,
# spread across possible regions
# Boston, Sydney, London, South Africa, Sao Paulo
fly scale count -y -r bos,syd,lhr,jnb,gru 10

  

Fly.io will do it’s best to equally spread the 10 instances out across the 5 regions there. BGP (aka “fancy DNS”, don’t @ me) will handle routing requests to the closest region, and the Fly Proxy will load balance across app instances within a region.

This is faster for users sending data to our application across the globe, and we gain higher availability by having more than one instance per region (each Fly.io region has a bunch of physical hosts, and Fly tries its best to distribute app instances across different physical hosts).

Auto Shut Down

A nice feature of Fly.io is that Machines can turn off while idle. To handle traffic spikes, we should carefully setup our fly.toml configuration to get the most out of this. Here’s what I would use, with the caveat that I totally made up the hard/soft limit concurrency settings (idk how many requests-per-second your app can handle):

    [http_service]
  internal_port = 8080
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 2
  processes = ['app']
  [http_service.concurrency]
    type = "requests"
    soft_limit = 100
    hard_limit = 150

[[http_service.checks]]
  grace_period = "10s"
  interval = "30s"
  method = "GET"
  timeout = "5s"
  path = "/up"

  

We’ve kept the default auto_(stop|start)_machines configuration (set to true). However we also kept the minimum machines running set to 2, so some Machines will always be on, even during low traffic periods.

We’ve also set concurrency settings - a soft and hard limit. These help the Fly Proxy know when to start spinning up extra instances and provide additional capacity. The values for these depend on how many concurrent requests your application can handle.

Lastly, we setup a health check so servers with issues can be taken out of rotation. This also helps with deployments.

Deployment

The default deployment strategy (with multiple machines) is “rolling” - one machine gets replaced at a time. Since we defined health checks, these are checked before each new Machine is replaced.

However, we can also use blue/green deployments! This creates a whole new set of Machines, checks their health, and then promotes them if/when they all pass. The old servers are then destroyed.

fly deploy --strategy=bluegreen

Blue/green is faster when you have many servers. While rolling swaps out servers one at a time, blue/green creates all new Machines concurrently. However it may fail deployments more often since the whole deployment reverts if any one of the servers fail a health check.

Blue/green is my prefered way to deploy.

Fly.io ❤️ Laravel

Fly your servers close to your users—and try scaling on Fly.io!

Deploy your Laravel app! →

Another Language?

This small use case (ingesting traffic and throwing the request into a queue) is actually a great opportunity to expand out of PHP (if that’s your thing).

A great language for this is Golang. It’s strict typing and relatively simple built-in HTTP support makes this type of use case really stable and fast. We’d get a faster bootup time and more throughput (requests per second) overall. Since it’s a small use case, the code shouldn’t (in theory!) get overly complex.

An example of Golang doing the same thing as our __invoke() controller method above is here.

Custom Laravel Pulse Cards

2024-02-13T00:00:00+00:00

Fly.io converts Docker images to fast-booting VMs and runs them globally. Fly.io supports Laravel! Check us out.

Pulse delivers a dashboard of application metric cards. Beyond these are reusable templates and a record-and-aggregation system to help us craft custom cards tailored to our very needs.

A Custom Requirement: Requests Per Machine

In a previous article, we configured our Fly Machines to run pulse:check to capture and display data for the Pulse servers card. This gave us insight into the CPU, Memory, and Storage usage of each of the Fly Machines serving our application:


Two Fly Machines’ server metrics are listed in Pulse’s `servers` card.

This time around, we want to display a card tallying requests received by each Fly Machine. Since Laravel Pulse doesn’t have a default card to display this information, we’ll have to create our own card for this.

And get this. We won’t have to create our card from scratch. Pulse enables us to create custom cards built on top of its Livewire Card component, and its data recording and aggregation system.

Let’s dive in!

Displaying a New Card

Pulse’s cards are built using Livewire. To display a custom Pulse card, all we have to do is create a new Livewire component that extends Pulse’s base Livewire Card component:

    /* app/Livewire/RequestsReceived.php */

namespace App\Livewire;

+  use Laravel\Pulse\Livewire\Card;

+  class RequestsReceived extends Card
{
    public function render()
    {
        return view('livewire.requests-received');
    }
}

  

In creating the corresponding view displaying our card, we’ll take plenty inspiration from Pulse’s available cards. We’ll use existing pulse blade components for a similar look and feel with the rest of the dashboard:

This blade component can receive parameters like cols and rows to determine the spatial division of the card.

class is also available to allow us to specify additional classes we’d want to map to the component.

    <!-- resources/views/livewire/requests-received.blade.php -->

<x-pulse::card :cols="$cols" :rows="$rows" :class="$class" >
    <x-pulse::card-header name="Request Distribution">
    </x-pulse::card-header>

    <x-pulse::scroll :expand="$expand">
        Display table Showing requests per Machine
    </x-pulse::scroll>
</x-pulse::card>

  

Finally, include this card in our pulse dashboard view so we can glimpse it:

    /*resources/views/vendor/pulse/dashboard.blade.php */

<x-pulse>
    <livewire:pulse.servers cols="full" />
+   <livewire:requests-received cols="full" />
</x-pulse>

  

Data to Display

Of course, the card currently has no data to display yet. Remember, we want info on the requests received by each machine. To display that information, we have to first get a hold of it.

We can retrieve this in two ways: using Fly.io’s metrics API or manually capturing requests and recording them using Pulse’s recording system.

That’s right! Fly.io provides access to metrics about our application running on its platform. We can query this info with a call to the Fly endpoint and display it in our Pulse Card. Here’s a repository which displays metrics from the Fly.io Metrics API in a custom Pulse card.

On the other hand, we can also capture requests received by our Fly Machines manually. Then we record these through Pulse’s data capture and aggregation system, then display that. Yup, that’s what we’re doing today.

Capturing Metrics with Pulse

To record requests per Machine via Pulse, we have to first intercept all requests, and capture the Fly Machine ID that receives each request.

To do so, we create a global middleware from where we install our recording logic:

    /* app/Http/Middleware/RecordRequest.php */

use Closure;
use Illuminate\Http\Request;
use Symfony\Component\HttpFoundation\Response;

+ use Laravel\Pulse\Facades\Pulse;

class RecordRequest
{
    public function handle(Request $request, Closure $next): Response
    {
+       $machineId = config(
          'pulse.recorders.'
          .\Laravel\Pulse\Recorders\Servers::class.'.server_name'
        );

+       Pulse::record('machine_requests', $machineId, null )
+       ->count();

        return $next($request);
    }
}

  

Above we first identify the Machine that received the request. We do this the same way Pulse identifies it: through the Pulse config server_name value. This would by default resolve in the Machine ID, which we pass to our $machineId variable.

We pass this along with two other attributes to Pulse’s record() method. “machine_requests” is what we’d want to identify the custom metric we want to record ( type of metric we want Pulse to keep track of ). $machineId identifies that the request was for a specific Machine ID ( key we can group entries of a specific type with ). Finally, null for the third to indicate no specific value to associate with the entry.

Looking at the repository of the Pulse package, it can be ascertained that a call to the Pulse::record() method eventually creates a new record in the pulse_entries table:

timestamp	type	key	key_hash	value
1706711201	machine_requests	4d8966da01e5d8	`<hash_key_here>`	null

Next, inspect the count() method which is chained to the statement. This will result in the creation/update of a row in the pulse_aggregates table which stores the computed count aggregate value associated with a combination of columns grouping our entry above:

bucket	period	type	key	key_hash	aggregate	value	count
1706712600	60	machine_requests	4d8966da01e5d8	`<hash_key_here>`	count	1	null

Notice there’re some similarly named columns in both tables above. Their type, key, and key_hash seem to hold the same values. However, their value columns differ quite greatly. The pulse_entries use it for the value attribute passed during the call to record(). While pulse_aggregates stores the result of an aggregation computation into value.

With this setup, every request will create( or update ) records in the pulse_entries and pulse_aggregates tables. Each new request per machine will be recorded as individual data points in the pulse_entries table. And thanks to count(), we get an easy-to-retrieve count aggregate values for each Machine’s request tally in the pulse_aggregates table. Finally, we can display this information in our Pulse Card.

Retrieving Pulse Metrics

We now have our requests recorded in our Pulse tables above. What we want next is to display the captured information in our custom Pulse card.

First, let’s retrieve our entries for our custom "machine_requests" metric type by calling Pulse's aggregate() method:

    /* app\Livewire\RequestsReceived.php  */

public function render()
{
      return view('livewire.requests-received',[
+          'machines' => $this->aggregate(
              'machine_requests', 
              ['count']
            )
      ]);
}

  

The aggregate() method receives two parameters. The first identifies the type of metric entries we’d want to retrieve, in which case we indicate a “machine_requests”. The second attribute lists out the aggregate values we want from each item in the retrieved list of entries. This for us is the count aggregate, as this will tell us the count of requests per machine at a given period of time.

We pass the resulting data received from the aggregate() call to a variable $machines accessible in our blade. This will allow us to display the list of Machines and their relevant request count in our corresponding view:

    <!-- resources/views/livewire/requests-received.blade.php -->

<x-pulse::card :cols="$cols" :rows="$rows" :class="$class" wire:poll.5s="">
    <x-pulse::card-header name="Request Distribution">
    </x-pulse::card-header>

    <x-pulse::scroll :expand="$expand">
        @foreach( $machines as $machine )
            {{ $machine->key }} = {{ $machine->count }}
        @endforeach
    </x-pulse::scroll>
</x-pulse::card>

  

Annd finally we get this:

Our very own, custom request-per-machine metrics, in our very own Pulse card! For a closer look into the different code snippets used above, do check out this sample repository here!

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

The Happenings Above

We needed a custom metric displayed in our Pulse dashboard, so we did a bunch of things.

First we captured requests by creating a global middleware and enclosing our recording logic on it using Pulse’s record() and count() methods. We then retrieved these metrics from Pulse’s database tables by calling Pulse’s aggregate() method. Finally, we displayed the information captured by extending Pulse’s custom Card component and making use of its existing blade components to follow after the default Pulse card theme.

With just three sections, we’ve got our very own custom Pulse metrics recorded to our needs, and displayed in our Pulse dashboard. Of course, this is a very basic rundown of the creation of Custom Pulse Cards. Please do read more about the different facets of the package’s custom cards from its official, wonderful documentation.

Monitoring Fly Machine Resource Usage with Laravel Pulse

2024-01-24T00:00:00+00:00

Fly.io converts Docker images to fast-booting VMs and runs them globally. Fly.io supports Laravel! Check us out.

Deploying an app to production may at the first few hours leave a sense of blindness on the state of our application. Questions like—Is every request going through smoothly? Will the servers suddenly run out of resource during some data crunching process?—run rampant like plaque in the back of our minds.

Often, we don’t get any indicator of frightening app-mishaps until we just do. Yikes!

Laravel Pulse is one of the Laravel ecosystem’s exciting new rollouts. It delivers a customizable dashboard of cards providing vital insights on application, queue, and server resource usage, as well as minute details on bottlenecks and exceptions.

Today, we’ll setup Laravel Pulse’s servers card in order to monitor and display resource usage of the Fly Machines running our Laravel app, and thankfully get visuals we can use to anticipate possible resource-mishaps event. Let’s get to it!

Prerequisites

Before anything else, we need our Laravel app deployed to a production environment. That can be quickly arranged by deploying it on Fly.io. This Laravel app should be connected to either a MySQL or Postgres database, as those are the only databases currently supported by Laravel Pulse.

Today we’re focusing on setting up our production application to use Pulse’s servers card feature. This means the Pulse package should also be installed into our app, with its configurable files made available( config, view, migration ) and its tables set up by publishing its assets and running the database migrations. Finally, its viewPulse authorization gate should be defined so the route to the pulse dashboard, /pulse, is accessible even in production. Our definition of this gate would determine whether or not a logged-in user can view the Pulse dashboard.

With all the above are properly accommodated, we should be able to access our Pulse dashboard:

Focusing on the Servers Card

That’s a bunch of cards above! But we only need one today, and that’s Pulse’s servers card. Pulse allows us to customize its dashboard thanks to its publishable view asset. Go ahead and revise it to only include our servers card:

/*resources/views/vendor/pulse/dashboard.blade.php */
<x-pulse>
    <livewire:pulse.servers cols="full" />
</x-pulse>

Here’s how our dashboard should finally look like:

Aha! And now, our dashboard’s looking pretty desolate. But, that’s only natural. There’s no data yet for the servers card to display. We have to first capture entries for this card.

Servers Card

Before capturing data for our card to display, let’s take a detour and look into what this servers card is, and, why should we even care. To do this, we’ll start off at the architecture our Laravel application was deployed in.

The next Fly Machine paragraph is a courtesy of the lovely @XE! Check out their articles at our blog section—they’ve got some amazing GPU recipes cooking in there

See, a Laravel app deployed in Fly.io runs on Fly Machines—Firecracker VMs bootable-stoppable in sub-second speeds. They are fast to spin up, fast to tear down, and can even stop themselves when our app isn’t being used, so we can save the costs of those Machines when they aren’t needed. In fact, we can configure our application to spin up its own Machines with the Machines API, but, that’s a story for another day.

By default, when launching a Laravel app( any app really! ) in Fly.io, two Fly Machines are automatically created and allocated for the app. This is convenient: we have a second Fly Machine as backup and a partner to the first. It can receive requests just like the first one, effectively sharing requests and processing burdens with the first one, and, ultimately, acting as fallback in case the first suddenly becomes unavailable.

Now, how do these two Fly Machines tie back to the Pulse servers card? We have two Fly Machines running our app, we’d like to get a quick glimpse of each Machine’s resource usage, because how would we know whether one or, even both, are consuming too much memory, CPU power, or running out of storage?

This is exactly where Pulse’s servers card comes into play—to monitor and display Fly Machine resource usage!

Capturing Resource Usage with `pulse:check`

Now that we’ve got the “why” setup, it’s time to finalize our setup by capturing data entries for the card.

To capture entries, all we have to do is run php artisan pulse:check on each Machine running our app( remember, we have two! ) and our resource usage shall be good for recording. Once started, the pulse:check command perpetually runs and periodically dispatches a SharedBeat event. The recorder for the servers card reacts to this event and records memory and CPU usage, and available storage.

Setting Up `pulse:check` on a Laravel Fly App

Since the pulse:check command runs perpetually once triggered, we only need to configure our Fly Machines to each trigger it once.

Setting that up is quite easy. See, during the time our app was initialized by Fly.io for deployment ( courtesy of fly launch ), a Dockerfile was auto-generated. This Dockerfile is used to setup the Fly Machine(s) that run our Laravel app.

This Dockerfile makes use of supervisor to run our app’s server. Thanks to this, we can use supervisor to handle running our pulse:check command as well. Simply create a new supervisor config file in the .fly directory generated by Fly.io, and configure it to run the pulse:check:

/* .fly/pulse-worker.conf */

[program:pulse-worker]
command=php /var/www/html/artisan pulse:check
autostart=true
autorestart=true
numprocs=1
redirect_stderr=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
stdout_events_enabled=true
stderr_events_enabled=true
user=www-data

Afterwards, update the autogenerated Dockerfile to copy this configuration into the /etc/supervisor/conf.d directory of our to-be Fly Machine:

    ## Dockerfile

/* OTHER PARTS OF DOCKERFILE HERE... */

+  # Setup CRON to update databases
+  COPY etc/pulse-worker.conf /etc/supervisor/conf.d/pulse-worker.conf

EXPOSE 8080
ENTRYPOINT ["/entrypoint"]

With those changes, push the changes to our Laravel Fly App by running fly deploy. Then behold, finally, a card to display resource usage of Fly Machines running our Laravel App:

Since we have two Fly Machines running our Laravel app, two rows are present above. Notice the first column, these are the IDs of the Fly Machines running our app( each is the name retrieved by pulse:check from each machine ). The next three columns respectively display the CPU, Memory, and Storage allocations consumed by each Machine during the most recent run of the pulse:check command.

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Take Away

In just a few steps, we now have our very own Pulse dashboard, customized to display resource usage of our Fly Machines. All we needed to set up was running the pulse:check command through a supervisor config file, revise the Dockerfile auto-generated by Fly to include this config file to the configs supervisord would run in the Fly Machines running our app. And just with that, we’ve set up Pulse to capture resource usage for each Fly Machine running our Laravel app!

Reusable Input Components without Livewire, with Livewire

2024-01-09T00:00:00+00:00

Fly.io converts Docker images to fast-booting VMs and runs them globally. Fly.io supports Laravel! Check us out.

If you prefer watching a video instead of reading an blog post: Dan Harrin has a video up on Laracasts that touches on the exact same topics as this article. Dan Harrin is of course the creator of Filament, so you will be in very good hands! Check out the video here: Laracasts video.

Having reusable input components is great. Knowing how to make them is even better! Follow along to see how we can create a generic TextInput component with a fluent interface that can be reused everywhere with just a few lines of code. While it won’t be a Livewire component, it can still make use of wire:model and other Livewire goodies. How is that possible? Read on to find out!

Character motivation

Before we dive in, let me explain why I won’t be using Livewire (kind of - you’ll see). Take a look at what I want to enable:

// example.blade.php
<div>
    {{ $this->getComponent() }}
</div>

    // Example.php
class Example extends Component
{
    public $state;

    // this is default
    public function render(): View|Closure|string
    {
        return view('components.example');
    }

    // this is what I'm talking about
    public function getComponent()
    {
        return TextInput('Cool Input')
            ->required()
            ->statePath('state')
            ->disabled(fn($state) => strlen($state) > 8);
    }
}

  

As you can see I can render a TextInput component on the fly by just returning the instance of the TextInput class I’ve just created. Using a fluent API I can configure it, and it can react to its own state. So, why aren’t we using Livewire components for this?

For this purpose, Livewire components have two major drawbacks:

Every component is an island. This means that sharing the data around will get overwhelming real quick.
You can’t create a Livewire component on the fly and then render it from a PHP class. Try it out: Inside of a PHP class (a Blade or Livewire component), create a new instance of a Livewire class and try to get it to render. Trust me, I tried and failed a lot. It doesn’t work.

Now that I’ve made my case against using Livewire components for this: let’s make this, shall we?

Building the component

Let’s start off with building a simple TextInput component. As mentioned, this will not be a Livewire component but a simple Blade component. Later on, we’ll see how we can still make use of Livewire goodies like wire:model.

Let’s make the component with php artisan make:component TextInput.

Add in a simple template:

    <div>
    <label for="{{$name}}" class="block font-medium text-sm text-gray-700">{{$name}}</label>
    <input id="{{$name}}" class="border-gray-300 focus:border-indigo-500 focus:ring-indigo-500 rounded-md shadow-sm">
</div>

  

As you can see, it’s quite simple: an input with a label grouped together in a div. For the input ID as well as the label text, I’m using a $name property that will be passed in using the view() method that’s used in the rendering of the component. Here’s a quick example:

    public function render(): View|Closure|string
{
    return view('components.text-input', ['name' => 'Example Name']);
}

  

Adding a fluent interface

Now, let’s focus on the php class of the component. I’d like it to enable a fluent interface so methods can be chained, just like Laravel’s query builder does for example. Here’s how that looks, as a reminder:

    // these methods (and others) can be chained to build up the query
$model::query()
  ->where('id', '>', 10)
  ->orderBy('created_at')

  

A fluent interface works by returning the current class in a certain method, so the next method in the chain receives the current class again.

Let’s start off with a make() method, that will leverage the class’s constructor to create a class instance. In the constructor, we want to add all the essential parameters to create an instance of this class. For now, that will just be a name for the component:

    class TextInput extends Component implements Htmlable
{

    protected string $name;

    public function __construct(string $name)
    {
        $this->name = $name;
    }

    public static function make(string $name) : static
    {
        // app(static::class) will use the constructor of static::class (so, this current class: 'TextInput') to create an instance of this class with whatever parameters are supplied.
        return app(static::class, ['name' => $name]);
    }

    public function render(): View|Closure|string
    {
        return view('components.text-input',
        [
            'name' => $this->getName(),
        ]);
    }

    public function toHtml()
    {
        return $this->render();
    }

    private function getName() : string
    {
        return $this->name;
    }
}

  

At this moment, we just pass a name into the component, and we use that $name as the label text and the ID for the input. To retrieve the name we use the private getName() method when rendering the component. You might be wondering why we don’t just make the $name property public, but I feel it adds too much clutter when configuring the component. I only want the methods you’re supposed to use when configuring the component to be public, nothing else.

For the Blade engine to understand how to turn this component class into html, we implement the Htmlable interface and with it, the toHtml() method. This will just call our render() method which is the same as using <x-component></x-component> in a Blade template.

To render the component on the TestPage page, we can use a getComponent() method to return the component as a whole. In there, we’ll make and configure the component. Check it out:

    class TestPage extends Component
{

    public function render()
    {
        return view('test-page');
    }

+   public function getComponent()
+   {
+       return TextInput::make('Test Input', $this);
+   }
}

  

Here’s the template for the TestPage:

    <!-- test-page.blade.php-->
<!-- I'm using the Breeze app layout here. -->
<x-slot name="header">
    <h2 class="font-semibold text-xl text-gray-800 leading-tight">
        {{ __('Testing Page') }}
    </h2>
</x-slot>

<div class="py-12">
    <div class="max-w-7xl mx-auto sm:px-6 lg:px-8">
        <div class="bg-white shadow-sm sm:rounded-lg">
            <div class="p-4">

                {{ $this->getComponent() }}
            </div>
        </div>
    </div>
</div>

  

{{ $this->getComponent() }} will use the toHtml() method in the TextInput class to turn the class instance into usable html. If we load this page, our component is displayed just like we wanted!

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Making the data reactive with Livewire

As of now our TestPage has no idea what’s inside the input, and it definitely can’t react to it. With Livewire, we’d simply add a wire:model onto an html input and that would be it. And that’s exactly what we’re going to do! Let’s turn the TestPage into a Livewire component first, by making it extend Livewire\Component instead of Illuminate\View\Component.

Now, we need to connect the input in the TextInput component to our TestPage Livewire component. How? Well, what if I told you the TextInput component will be rendered completely as dumb HTML and dropped into the template of the TestPage Livewire component?

So, if we were to put a wire:model in that chunk of HTML that is our Blade component… That’s right, we can use wire:model inside Blade components! This will work as long as it’s rendered inside of a Livewire component.

One caveat: we need to make sure our Blade component and our Livewire component agree what property gets used for the wire:model. For that, I’ll add a $statePath variable on the TextInput Blade component and pass it in when we create the component:

    class TestPage extends Component
{
+   // this will contain what the user entered in the TextInput component.
+   public string $state;

    public function render()
    {
        return view('livewire.test-page');
    }

    public function getComponent()
    {
        return TextInput::make('Test Input', $this)
+           ->statePath('state'); // this lets the TextInput know what to put in wire:model=""
    }
}

  

    class TextInput extends Component implements Htmlable
{
    protected string $name;
+   protected string $statePath;

    // __construct() and make() method here, they are unchanged

+   public function statePath(string $statePath) : static
+   {
+       $this->statePath = $statePath;
+       return $this;
+   }

    public function render(): View|Closure|string
    {
        return view('components.text-input',
        [
            'name' => $this->getName(),
+           'statePath' => $this->getStatePath(),
        ]);
    }

    // toHtml() and getName() are also unchanged.

+   private function getStatePath() : string
+   {
+       return $this->statePath;
+   }
}

  

    <!-- text-input.blade.php -->
<div>
    <label for="{{$name}}" class="block font-medium text-sm text-gray-700">{{$name}}</label>
-   <input id="{{$name}}" class="border-gray-300 focus:border-indigo-500 focus:ring-indigo-500 rounded-md shadow-sm">
+   <input id="{{$name}}" class="border-gray-300 focus:border-indigo-500 focus:ring-indigo-500 rounded-md shadow-sm" wire:model="{{$statePath}}">
</div>

  

I’ll quickly add a debugging line in the Livewire TestPage template, to show that the wire:model works. Just add this line somewhere in your Livewire component:

<span>Wire:model property = {{$state}}</span>

To see the data binding in (almost) real-time, change the wire:model in the TextInput template to wire:model.live . Try it out by typing in the text box, the value should be printed out on the page!

Enabling and disabling the input

Now let’s see how far we can take this. Usually, form inputs can be disabled on certain conditions. Let’s see if we can enable or disable the TextInput blade component from our Livewire page. Let’s begin with passing in a boolean variable from the Livewire page to disable/enable the Textinput:

    class TextInput extends Component implements Htmlable
{
    protected string $name;
    protected string $statePath;
+   protected bool $disabled;

    // unchanged methods here...

+   public function disabled(bool $disabled) : static
+   {
+       $this->disabled = $disabled;
+       return $this;
+   }

    // more unchanged methods...

+   private function isDisabled() : bool
+   {
+       return $this->disabled;
+   }
}

  

We’ll need to update the template as well to show the disabled state:

    <div>
    <label for="{{$name}}" class="block font-medium text-sm text-gray-700">{{$name}}</label>
-   <input id="{{$name}}" class="border-gray-300 focus:border-indigo-500 focus:ring-indigo-500 rounded-md shadow-sm" wire:model="{{$statePath}}">
+   <input id="{{$name}}" class="border-gray-300 focus:border-indigo-500 focus:ring-indigo-500 rounded-md shadow-sm disabled:bg-gray-100"
+          @disabled($disabled) wire:model.live="{{$statePath}}">
</div>

  

If we now add a $disabled property on the Livewire page along with a toggle button, and pass it to the Textinput component, we’ll see that we can toggle the component on and off:

    class TestPage extends Component
{
    public string $state;
+   public bool $disabled = false;

    public function render()
    {
        return view('livewire.test-page');
    }

    public function getComponent()
    {
        return TextInput::make('Test Input', $this)
            ->statePath('state')
+           ->disabled($this->disabled);
    }

+   public function toggleDisabled()
+   {
+       $this->disabled = !$this->disabled;
+   }
}

  

For reference, here’s what the toggle button on the Livewire page template looks like:

        <button class="px-4 py-2 bg-blue-700 rounded-md font-semibold text-white hover:bg-blue-800" wire:click="toggleDisabled">
        Toggle
    </button>

  

By toggling the $disabled property, the Livewire component will get re-rendered. During the re-rendering, the getComponent() method will be called again, so the disabled() method will now receive the updated value of the $disabled property of the Livewire page. The Blade component itself is not reactive, but it feels reactive because the Livewire page will re-render the component whenever a public property of the Livewire page is updated.

In the same way we can pass a Closure to the $disabled property, and toggle the disabled state based on its input. Check it out:

    class TextInput extends Component implements Htmlable
{
    protected string $name;
+   protected \Livewire\Component $livewire;
    protected string $statePath;
-   protected bool $disabled;
+   protected bool|Closure $disabled;

-   public function __construct(string $name)
+   public function __construct(string $name, \Livewire\Component $livewire)
    {
        $this->name = $name;
+       $this->livewire = $livewire;
    }

-   public static function make(string $name) : static
+   public static function make(string $name, \Livewire\Component $livewire) : static
    {
-       return app(static::class, ['name' => $name]);
+       return app(static::class, ['name' => $name, 'livewire' => $livewire]);
    }

    // some unchanged methods here...

-   public function disabled(bool $disabled) : static
+   public function disabled(bool|Closure $disabled) : static
    {
        $this->disabled = $disabled;
        return $this;
    }

    // more unchanged methods here...

    private function isDisabled() : bool
    {
-       return $this->disabled;
+       return $this->evaluate($this->disabled);
    }

+   private function evaluate($value, array $parameters = [])
+   {
+       $state = data_get($this->livewire, $this->getStatePath());
+       if ($value instanceof Closure) {
+           return app()->call($value, array_merge($parameters, ['state' => $state]));
+       }

+       return $value;
+   }
}

  

The main update here is the evaluate() method. This enables us to pass a Closure into the disabled() method, which we can then evaluate at runtime. To do that we need to get the value of the Livewire property that’s connected to our input. We can do that by passing our Livewire parent into our constructor as $livewire, and then using data_get($this->livewire, 'state').

This retrieves the $state variable on the Livewire component. If our $disabled property is a Closure, we’ll call it using app()->call() and make the $state variable accessible, as well as the $parameters we defined as a function parameter. That way if the Closure uses a variable called $state, the contents of the $state variable we defined on the Livewire page will be passed in. Since we used wire:model to connect that variable to our input, we’ll receive the input value. Neat! This enables us to disable our input based on the input value, for example if it exceeds 8 characters:

    // TestPage.php
public function getComponent()
{
    return TextInput::make('Test Input', $this)
        ->statePath('state')
        ->disabled(fn($state) => strlen($state) > 8);
}

  

Displaying errors

But wait, there’s more!™️

Livewire has more tricks than just wire:model! Let me show you that validation also works great here. I’ll quickly add a validation rule on $state, like this:

    +   #[Validate('required|max:8')]
    public string $state;

If we were to type a string of 9 characters in the input field, we wouldn’t see anything. I assure you the error is being thrown, though! We just need to show it on the Textinput component:

    <!-- text-input.blade.php -->
<div>
    <label for="{{$name}}" class="block font-medium text-sm text-gray-700">{{$name}}</label>
    <input id="{{$name}}" class="border-gray-300 focus:border-indigo-500 focus:ring-indigo-500 rounded-md shadow-sm" wire:model="{{$statePath}}">
-   <input id="{{$name}}" class="border-gray-300 focus:border-indigo-500 focus:ring-indigo-500 rounded-md shadow-sm disabled:bg-gray-100"
-         @disabled($disabled) wire:model.live="{{$statePath}}">
+   <input id="{{$name}}" class="border-gray-300 focus:border-indigo-500 focus:ring-indigo-500 rounded-md shadow-sm disabled:bg-gray-100"
+         @disabled($disabled) @error($statePath) invalid @enderror wire:model.live="{{$statePath}}">
+   @error($statePath)
+     <span class="text-red-700 text-sm>{{ $message }}</span>
+   @enderror
</div>

  

As you can see, the @error() Blade directive works great together with Livewire validation. The logic is the same as for the reactivity: our component is just a building block used in a Livewire component, so it ‘inherits’ the Livewire goodness form its parent.

And there we have it: a fully reusable TextInput component! Thanks to the fluent interface it’s easy to use and feels native to Laravel. On top of that, it integrates perfectly into Livewire which makes reactivity a breeze. Thanks for reading and I’ll see you in the next one!

Autoscaled Queue Workers

2023-12-14T00:00:00+00:00

Fly.io converts Docker images to Micro VMs and runs them globally. Fly.io supports Laravel! Check us out.

Fly Machines let you programmatically create and manage fast micro VMs. They’re great for scaling! One thing you might want to scale is queue workers.

I made a thing™ that can do this, automatically.

The Goal

I wanted to do the least amount of work, but still make a resilient system.

This means threading the needle between complex clustering logic and something too-simple-to-actually-work. But then I remembered this is PHP, so “stupid but effective” is the way to go. Obviously, we use CRON.

The things I wanted this to do were:

Have something do all of this work automatically
Scale up Machines (VMs) that immediately start churning through queue jobs
Allow for some number of “base” queue workers (always on and running)
Logic to detect if we need to scale up
The ability to scale down

Here’s how I did those things.

Doing the Work

I had Laravel control the scaling. It is PHP afterall, and if there’s one thing PHP devs love to do, it’s use PHP for literally everything. (I’m allowed to make fun of PHP devs because I’ve learned other languages). (That was sarcasm). (But really, other languages can be cool too).

The poor stooge in charge of all this is a Laravel console command. The actual command is here, but we’ll talk about the highlights below.

php artisan make:command --command=fly:work FlyWorkerCommand

I threw this command into the scheduler, so the almighty CRON can power it. Once a minute, this command runs and decides if we need to do some work.

    // File: app/Console/Kernel.php
protected function schedule(Schedule $schedule): void
{
    $schedule->command('fly:work')
        ->onOneServer()
        ->everyMinute();
}

  

If you’re like Jack and need clarification, I used onOneServer() as well so this command only ever runs in one place, to remove race conditions. In my case I’m using Redis for session, cache, and queues. Using Redis for cache is what matters most here - Laravel uses the Cache system for locking. You need to use some central cache mechanism that all server instances read from to use onOneServer() effectively (redis, database, probably others but I’m not opening a browser tab to find out right now).

Machines to Run Queue Workers

We can run php artisan queue:listen for a queue worker since Laravel’s queue worker is just part of the framework. To run more workers, we just need to make more copies of our application VM!

I’m running my app on Fly.io, which means a Docker image was generated and pushed up to the Fly Registry. Fly uses that image to make a VM. We can tell Fly to make any number of VM’s anytime we want!

So we have the pieces needed to scale up workers. Any new queue worker Machine (VM) we create can re-use our existing image. All we need to do is tell it to run artisan queue:work instead of spin up the web server stuff.

To run the artisan queue:work command, we overwrite the Docker CMD used when starting a container (or VM in our case). This is standard “How to Fly” stuff - we do exactly this to run CRON and Queues on Fly.io.

All we’re adding on top of this is some logic to spin up multiple worker VM’s.

Creating a Machine

Let’s see what it looks like to actually create a Fly Machine that will run as a queue worker.

No need to prattle on about it, here’s some code sending an HTTP request to the Fly API:

    $appName = "your-fly-app-name-here";

Http::withToken($this->api_key)
    ->asJson()
    ->acceptJson()
    ->baseUrl("https://api.machines.dev/v1/apps/".$appName)
    ->post('/machines', [
        "region" => env('FLY_REGION'),
        "config" => [
            "image" => env('FLY_IMAGE_REF'),
            "env" => ['foo' => 'bar',], 
            // false for base machines, true for scaled Machines
            "auto_destroy" => false, 
            "guest" => config('fly.vm'), // shared, 1 CPU, 1024M ram
            "init" => [
                // queue:listen on base worker Machines
                // queue:work --stop-on-empty for 
                //   scaled worker Machines
                "cmd" => ["php", "artisan", "queue:listen"],
            ],
            "metadata" => [
                "fly_laravel_queue_machine" => "base", // or "scaled"
            ],
            // restart for base only
            // else policy = "no" for scaled Machines
            "restart" => [ 
                "max_retries" => 3,
                "policy" => "on-failure",
            ],
        ],
    ]);

  

This is a pretty standard Machine-creation API call. There’s a few things to note!

First, I’ll assume this API call is sent within a Laravel app running in a Fly VM. This means environment variables FLY_REGION and FLY_IMAGE_REF are available. We make use of those to tell our new Machine to run in the same region and re-use the current Docker image, so it runs the worker Machines with the same code.

We also add some metadata to differentiate worker Machines (“base” and “scaled”) vs ones that are managed by flyctl (the web app and CRON Machines).

Finally the init section is used to tell the VM to run php artisan queue:listen instead of doing the default thing of spinning up a web server.

Our code will create n base Machines (configurable). Then we’ll (later) create some scaling logic to scale up past those “base” Machines if our queue passes some threshold we set.

When to Create the Always-On Base VM’s

The fly:work command does a bunch of work (duh). The first thing it does is find out what Machines exist within the current Fly app.

This then gives us the opportunity to see which Machines need to be created (if any).

Here’s some logic on creating “base” Machines - the queue workers we always want working no matter how many queue jobs are pending.

    $api = new MachineApi(config('fly.app_name'), config('fly.api_key'));

// We pass MachinePool a list of all the Machines in our app
// MachinePool then does some filtering on API results for us
// to differentiate between web app, cron, base workers
// and scaled worker Machines
$pool = new MachinePool($api->listMachines());

// Destroy base workers if we have too many
if ($pool->baseMachines->count() > config('fly.min_workers')) {
    $getRidOf = $pool->baseMachines->count() - config('fly.min_workers');
    for($d=0; $d<$getRidOf; $d++) {
        $api->destroyMachine($pool->baseMachines[$d]['id']);
    }
}

// Create base workers if needed
if ($pool->baseMachines->count() < config('fly.min_workers')) {
    $neededMachines = config('fly.min_workers') - $pool->baseMachines->count();
    for($i=0; $i<$neededMachines; $i++) {
        $api->createMachine([...]);
    }
}

  

We get our state of the world (trusting in the API call to list Machines) and then decide if we need to delete base Machines or add base Machines.

We have logic to delete base machines if we change configuration, for example if we had 4 base workers and later decided we only need 2.

The actual code has a few more sanity checks for data integrity (read: a bunch of isset()-style checks), but nothing too crazy.

When to Scale Up

We need some logic telling us when to scale up. I made it so you can write your own scaling logic, and have a default scaling technique that is just dumb enough to work.

I decided I wanted to set a threshold of “queue jobs per machine”. Once that threshold gets too high, we’d scale up queue workers.

I started with a threshold of 10 queue jobs per worker. How’s that work?

Let’s say we suddenly have 100 queue jobs and 2 (base) workers - that’s 50 jobs per machine, well over our threshold! To get down to 10 jobs per worker, we’d need to scale up 8 additional workers (for a total of 10). We’ll then have 100 jobs spread over 10 machines, which gets us down to 10 jobs per worker.

Here’s the JobsPerWorker class, which does the math to decide how many extra worker Machines we should make:

    <?php

namespace App\Fly\Scalers;

use App\Fly\MachinePool;
use Illuminate\Contracts\Queue\Factory;
use App\Fly\Scalers\ShouldScaleInterface;

class JobsPerWorker implements ShouldScaleInterface
{
    public function __construct(
        protected Factory $queue,
        protected MachinePool $pool,
        protected int $jobsPerMachine = 10
    ){}

    /**
     * @return int Number of Machines to create
     */
    public function shouldScale(
      ?string $connection=null,
      ?string $queue=null
    ): int
    {
        $totalManagedMachines = $this->pool->managedMachines->count();

        // If we don't have base machines yet (likely on first run)
        // then we don't do any scaling yet
        if($totalManagedMachines == 0) {
            return 0;
        }

        // jobs per machine = total job count / total worker machines
        $currentJobsPerMachine = 
            $this->queue->connection($connection)->size($queue) 
            / $totalManagedMachines;

        if ($currentJobsPerMachine > $this->jobsPerMachine) {
            // return the number of machines 
            // we should scale up by, if any
            return 
                (ceil($currentJobsPerMachine / $this->jobsPerMachine)) 
                  - $totalManagedMachines;
        }

        return 0;
    }
}

  

Laravel has a way for us to get the count of jobs on any given connection/queue. We use this to help us do some math!

The “managed machines” is the count of workers (base + scaled) we currently have. We scale up additional workers (if needed) if we meet the threshold of $currentJobsPerMachine / $totalManagedMachines (if that number is greater than 10, in our example).

Creating Scaled Queue Workers

The same (ish) API call we saw above can be used to create scaled workers. The main differences for “scaled” machines are that we use queue:work instead of queue:listen, and we set auto_destroy to true.

Why the difference?

The queue:listen command is meant to stay alive - it actually runs queue:work behind the scenes. On the other hand, queue:work is meant to be more short-lived and is able to just process 1+ jobs and then stop. We want it to stop, since that tells the VM to stop and autodestroy itself. Scaling down comes for free!

Here’s some of the logic from the fly:work command:

    $connection = null; // default queue connection
$queue = ''; // default queue
$scalingClass = config('fly.scale_controller'); // e.g. JobPerWorker

// Result of our math from above
$numberMachinesNeeded = $pool
  ->getScaler(config('fly.scale_controller'))
  ->shouldScale($connection, $queue);

for($j=0; $j<$numberMachinesNeeded; $j++) {
    $api->createMachine([
        // I use config() in reality, instead of env() here
        "region" => config('fly.region'),
        "config" => [
            // yep, more config()
            "image" => config('fly.image'),
            // We can use use env from 
            // the web app VM!
            "env" => $pool->appMachines->first()['config']['env'], 
            "auto_destroy" => true, // true for scaled machines
            "guest" => config('fly.vm'),
            "init" => [
                "cmd" => [
                  "php", "artisan", 
                  "queue:work", "--stop-when-empty"
                ],
            ],
            "metadata" => [
                "fly_laravel_queue_machine" => "scaled", // vs "base"
            ],
            // restart is for base Machines only
            "restart" => [
                "policy" => "no",
            ],
        ],
    ]);
}

  

Nothing too crazy there! I moved the env('FLY_REGION') calls to a config/fly.php file so I could make config() calls - which helps for local dev.

I also showed how I got the env populated - I grabbed the environment set for the web application machines.

Finally I set a slightly different metadata, cmd, and restart policies for the scaled machine.

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Scaling Down

As I hinted above, I don’t have logic for scaling down workers! Instead, I let the Machines do that automatically for us.

Fly Machines can destroy themselves when whatever it’s running exits. The workers are running php artisan queue:work. We can add a handy flag to have it stop itself: --stop-when-empty.

This means when we’ve scaled up enough, and our queue becomes empty, the queue worker will stop, the Machine will exit, and destroy ourselves.

This actually isn’t perfect, but it’s pretty workable. There is a fun edge case of when a queue is never empty, but also doesn’t have enough jobs to warrant all those extra scaled-up workers.

Fixing that would involve a few potential things:

A “sidecar” (second) process running in the scaled up queue workers that shuts it down after a certain period of time
A max number of jobs the scaled workers are allowed to run before stopping (--max-jobs flag)
Scaling logic that calculates a moving average of the number of pending jobs (the “queue depth”) and using that number against our configured threshold. Then it can decide to add or delete worker scaled worker VM’s.
- This means adding scale-down logic, instead of letting scaled workers stop themselves

Option 3 is probably technically the best, but it’s more complicated(ish) to do properly - you need to store data somewhere and keep tabs on the number of jobs pending on every run to do the calculation. Idea number 2 is better than the original, and is not a terrible solution, but still not perfect. Idea number 1 is kinda annoying to setup tho, so I think I’m overall in favor of idea 3.

Either way, implementing that is an exercise for the reader (for now).

Streaming updates with Livewire and Fly Machines

2023-11-28T00:00:00+00:00

Need a place for your Laravel app in the cloud? Fly it with Fly.io, it’ll be up and running in minutes!

Previously, we looked into displaying progress in bulk processing of data. Just with one request, we were able to stream updates to the UI as each data completed processing thanks to the wire:stream directive. This allowed the UI to get updated real-time as each item updated in the server:

When the client sends a request to the server expecting a streamed response, that long-lived request is going to take up connection space, resource, and time from our web app.

What if our web app starts struggling with all our long-lived requests that pile up with each new request from our users? It’s going to start becoming unresponsive and unavailable.

Wouldn’t it be neat then, if instead of our web app’s instance—which we expect to be responsive and available at all times—instead of that instance, we have a separate instance, exclusively handling long-lived requests that wait on streamed responses?

That would be neat, right?! And yes, we can have exactly that extra processing power—on the go—with Fly Machines—lightweight, Firecracker VMs startable and stoppable at subsecond speeds.

—Excited?!

Then, let’s do this!

Prerequisites

In a previous article, we made use of Livewire’s wire:stream directive to get up and running with streamed responses sent to the client. We’re going to build on top of that setup, but of course, it’s not a necessary arrangement. You can have any kind of wire:stream setup for your Laravel app, have it deployed on Fly.io, and you’ll be good to go.

Livewire’s `wire:stream`

Say, we have a Livewire component, and in it, a button wired to a process() method of the component. This method does a bunch of processing on a list of data, and thanks to Livewire’s stream() method, it sends updates to the client as each item completes:

    /* app/Livewire/MyComponent.php */

public function process(){
  foreach( $this->ids as $id=> $details ){
      // Process item
      $this->processItem( $id );      

      // Update item's status to processed       
      $this->ids[ $id ][ 'status'] = 'Processed!';

      // Stream update to the client
      $this->stream(
          to: 'process-stream'.$id,
          content:  $this->ids[ $id ][ 'status'],
          replace: true
      );
  }
}

  

The first time Livewire’s stream() method is called, Livewire streams back a long-running response to the client. As each item is processed, Livewire sends each item’s update to this stream. Each update is then reflected to its matching wire:stream-ed element in the client identified by the stream name process-stream<n>:

    <!-- app/resources/livewire/my-component.blade.php -->

@foreach( $items as $key=>$item )
    <tr>
        <td wire:stream="process-stream{{ $item->id }}">
            {{ $ids[$item->id]['status'] }}
        </td>
    </tr>
@endforeach

  

This long-running request stays open until all items in the list are completely processed and the process() method’s call ends.

Re-assigning Long-Running Requests

Right now, the request made to call the process() method is sent to the main instance of our web app. That instance is in charge of handling user-facing requests. So as much as possible, we’d want it to be available and free from long running requests, such as the request to our process() method.

To keep our web app’s instance hands-free from requests such as our streamed response above, we would have to redirect the requests to a separate instance, created specifically to handle our long-running requests.

Take notice! There are three note-worthy things above:

Catch long-running requests,
Create an instance to handle these requests, and
Redirect requests to the new instance.

Catching Requests

To redirect a request to our Livewire component’s process() method, we’ll have to catch requests first. Run php artisan make:middleware RedirectRequestToTaskMachine, this will create a middleware we’ll use to intercept and redirect our process() requests.

The first part of our middleware is simple: check for a Livewire request to a process() method, indicated from request->components[n]['calls'][m]['method'] == 'process':

    /* app/Http/Middleware/RedirectRequestToTaskMachine.php */

public function isForTaskerMachine( Request $request )
{
  // Check request is for Livewire "process()" method
  if( isset($request->components) ){
    foreach( $request->components as $component ){   
        if( isset($component['calls']) ){
            foreach( $component['calls'] as $call ){
                if( $call['method'] == 'process' ){
                    Log::info('process method request found!');
                    return true;
                }
            }
        }
    }
  }
  // Not a request to "process()"
  return false;
}

  

The trick here is to name our long running tasks using one name( in our case, process() ), this way we can always catch long-running Livewire requests through this method name!

Creating the New Instance

Our next puzzle piece is the creation of a new instance to handle processing our items. It will have all capabilities required to handle our Livewire’s wire:stream processing, and exclusively take-in long-running requests. To provide these capabilities easily, we’ll create a near-exact copy of our web app’s instance( and therefore provide the needed capabilities! ).

With our Laravel app deployed in Fly.io, we can easily set this up with Fly Machines: First, we set an array of configuration values, then pass this array to an api call that will spin and boot up a new Fly Machine.

We can create our Fly Machine with the following configuration array:

Here’s a reference, you’re welcome:

Attribute	Description	Value Passed
config.image	Fly Machines are built from Docker images! So we pass one.	`FLY_IMAGE_REF` - Since we want to create a copy of our web app we provide the image used to create it. `FLY_IMAGE_REF` is an env variable automatically set by Fly.io pointing to the image reference used to spin up our app’s Fly Machine.
config.env	Environment variables required by our app.	We include all env variables necessary for our web app to work on our new Fly Machine. We also include a new variable “`FLY_TASKER_MACHINE`” as a flag that this new Machine can be used for processing long running tasks.
config.services	Services enable connections from the internet.	Ports for https, http, and most importantly, the `autostop` flag to stop our VM when it goes idle.

    public function getPayload()
{
    return  [
        "config" => [
              // Image to build our Machine From
              "image" => env('FLY_IMAGE_REF'), 

              // Env Vars required by our app
              "env" => [
                  // ... Other env vars
                  "FLY_TASKER_MACHINE"=>"true"
              ],

              // Enable connection from the internet 
              "services" => [
                    [
                        "autostop"=>true,
                        "ports" => [
                          [
                              "port" => 443, 
                              "handlers" => [
                                "tls", 
                                "http" 
                              ] 
                          ], 
                          [
                            "port" => 80, 
                            "handlers" => [
                                "http" 
                            ] 
                          ] 
                        ], 
                        "protocol" => "tcp", 
                        "internal_port" => 8080 
                    ] 
              ], 
        ] 
      ];
}

  

There’s a bunch of stuff up there! Let’s break it down with some context.

See, every application we deploy in Fly.io is, at its core, a Fly Machine—and yes—this includes the Laravel web app we deploy in Fly.io. That main instance of our web app? That’s a Fly Machine!

Fly Machines are Firecracker VMs that are bootable-stoppable in subsecond speed! Each Fly Machine is built out from a Docker image, and accepts extra configuration settings like environment variables and services(amongst other things!).

Fly Machines in stopped state don’t use RAM or CPU—no additional costs are incurred for that stopped state duration!

This is why we have the first attribute config.image—pointing to FLY_IMAGE_REF, the docker image of our web app’s Fly Machine. As well as config.env to pass all env variables used by our web app, with an additional flag FLY_TASKER_MACHINE that we’ll use to mark the machine as a “Tasker Machine”. Finally, we include a config.services to allow public access to the new Machine, and importantly include an autostop flag to allow the Fly proxy to automatically stop it once it becomes idle.

Once we have the configuration figured out above, we simply send this to the Fly Machine’s api endpoint to create a new Fly Machine on the spot:

    public function createTaskerMachine()
{
  // The URL is built with the `FLY_APP_NAME` env set by Fly.io
  $url = 'https://api.machines.dev/v1/apps/'. env('FLY_APP_NAME') .'/machines';

  // Please retrieve your token with `fly auth token`
  $token = env('FLY_AUTH_TOKEN');

  // Create Task-Processor Fly Machine
  $response = Http::withToken( $token )
  ->post(
      $url, 
      $this->getPayload()
  );

  $result = $response->json();
  return $result['id'];
}

  

The function above will, on a successful creation of the Fly Machine, return the value of id assigned to the Fly Machine we just created. With it we can direct our requests to our newly created Fly Machine—which we shall now refer to, and dub as, “Tasker-Machine”!

Finding the Tasker-Machine

We can keep creating Tasker-Machines on the go—this will create one new Machine per long running request, at which case we’ll have to clean up after each Machine completes its intended processing.

OR instead, we can use the first Tasker-Machine to handle all our long-running request! We simply inspect our available Machines, and get the Machine which contains the FLY_TASKER_MACHINE flag:

    public function getTaskerMachine( )
{
    // The URL is built with the `FLY_APP_NAME` env set by Fly.io
    $url = 'https://api.machines.dev/v1/apps/'. env('FLY_APP_NAME') .'/machines';

    // Please retrieve your token with `fly auth token`
    $token = env('FLY_AUTH_TOKEN');

    // Get list of machines
    $response = Http::withToken( $token )->get( $url );
    $result = $response->json();

    // Return Machine with FLY_TASKER_MACHINE flag
    foreach( $result as $machine ){
      if( isset($machine['config']['env']['FLY_TASKER_MACHINE']) ){
          return $machine['id'];
      }
    }

    // Otherwise, Create the Tasker Machine
    return $this->createTaskerMachine();
}

  

The function above makes a call to the Fly Machine endpoint to retrieve the list of Machines for our app. It then loops through this list to find the Machine containing the Tasker-Machine’s flag and return that Machine’s id. If that’s not available, we call the createTaskerMachine() method to create a new one, and return that new Fly Machine’s id as its result.

Replaying to the Tasker-Machine

Now that we’ve setup methods isForTaskerMachine(), and getTaskerMachine() of our middleware available, it’s time to orchestrate them together to redirect our requests to the Tasker-Machine:

    public function handle(Request $request, Closure $next): Response
{ 
    // Req should be processed by Tasker-Machine
    if( $this->isForTaskerMachine( $request ) ){

        // The current Fly Machine is a Tasker-Machine 
        if( env('FLY_TASKER_MACHINE') != null ){

            // Go ahead and proceed with request
            return $next($request);

        }else{
        // The current Fly Machine is NOT a Tasker-Machine

            // Get machine id of the TaskerMachine
            $instanceId = $this->getTaskerMachine(); 

            // Stop, and redirect request to Tasker Machine
            return response('', 200, [
              'fly-replay' => 'instance='.$instanceId,
            ]);

        }

    }
}           

  

Above, we first check if the request should be redirected to the Tasker-Machine. If so, we check what kind of Fly Machine is handling the request through the env flag, FLY_TASKER_MACHINE. It’s availability tells us that the current Machine handling the request is the Tasker-Machine, and therefore we proceed with handling the request.

On the other hand, if it’s not available, that means the handler is our app’s main Fly Machine. We can’t continue the processing from here, so we get the Tasker-Machine’s id. With this, we stop the processing the request, and send an abrupt response back to the client.

This response holds the essential fly-replay response header pointing to the newly created machine’s id. This let’s Fly.io’s proxy network know that we’d like to “replay” the request to a different Fly Machine, specifically the one with the id indicated in the header.

Replaying to the Main Fly Machine

Right now we have two Fly Machines accepting user requests. The main, original Machine that was spun up when we deployed our Laravel app to Fly.io, and the other, “Tasker-Machine” Machine.

This means that our user requests have the chance to get processed by the Tasker-Machine. We don’t want that! We want our Tasker-Machine to handle our specific long-running requests, BUT, have our main Fly Machine exclusively handling all other requests.

We already have our logic above for redirecting requests to the Tasker-Machine, let’s setup our logic for redirecting requests to the original Fly machine:

    public function handle(Request $request, Closure $next): Response
{ 
    // Check if request is applicable for Tasker-Machine
    if( $this->isForTaskerMachine( $request ) ){
      // ...Redirect to Tasker-Machine Logic here...
+   }else if( env('FLY_TASKER_MACHINE')!= null ) {
+      // Replay to the main Fly Machine
+      return response('', 201, [
+          'fly-replay' => 'elsewhere=true',
+      ]);  
+    }

}

  

Above, we simply add an else if after the first condition that checks whether the request is for the Tasker-Machine. Because if the request is not for the Tasker-Machine, and it lands on the Tasker-Machine, evidenced by env('FLY_TASKER_MACHINE')!=null, that means we need to redirect it to the main Fly-Machine.

Again we make use of the fly-replay response header to redirect our request. This time around we use the value of elsewhere=true, which asks the Fly proxy to replay the request to a Fly Machine other than the current Machine.

Applying the Middleware

Now that our re-direction logic is complete, it’s time to apply this middleware to our requests. We do this by applying our middleware as a global middleware in our Kernel.

Once applied, all requests will pass through our middleware, and properly get routed to the correct machine.

Here’s the logs when our Main Fly Machine gets a process() method request, the Tasker-Machine is not yet available, so a new Tasker-Machine is created:

Once the new Machine is available, it receives the redirected request thanks to Fly.io’s fly-replay feature:

And see our process() method handled by our new Tasker-Machine!

Look Forward!

And, that’s it! We have two Fly machines. We created one on the go ( Tasker-Machine ), which handles a specific kind of requests for us. While we have our original Machine ( created during deployment of our app )handling all other requests.

Fly Machines provide such versatile a playing field for our Laravel apps. It can be created on the go, isolate user code, and give us an option to provide different “playing fields” for different services of our app.

We can create as much Fly Machines to handle as much different parts of our Laravel app! We can isolate resources for long-running tasks( like we just did! ). We can isolate user code( even untrusted ones! ). We can even run our background workers with Fly Machines!

And remember that little small detail about the autostop part of our configuration? Yup! Stopped Machines don’t use CPU or RAM, and so don’t incur additional processing costs at its stopped state!

Such an exciting piece of callable VMs aren’t they?

Go ahead then, and check out our Fly Machines!

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Making the CLI and Browser Talk

2023-10-31T00:00:00+00:00

Need a place for your Laravel app in the cloud? Fly it with Fly.io, it’ll be up and running in minutes!

The CLI and web browser don’t really talk to each other. Let’s force some communication and make it 🎃 wyrd 🎃.

We’re going to let users create an account (“register”) in our app, using the CLI and their web browser.

CLI as User Interface

The flyctl (aka fly) command is the main UI for Fly.io. There’s nothing special about how it works tho - to create servers, deployments, and everything else, it’s “just” making calls to the Fly API. This requires an API key.

The fly command can get that API key for you automatically. Getting the API token happens on registration or login (fly auth signup|login). The command kicks you out to the browser. After you sign up and return to your terminal, flyctl knows you’ve authenticated and already has a new API key.

How does flyctl know you’ve done that, and how does it get the API key?

Let’s see how (and do it ourself).

The Magic

There’s no magic, not even a hint of 🧙‍♀️ witchcraft.

The signup command gets a session ID, and performs an extremely polite-but-boring polling of the API (using that session ID) to ask if the API could please inform us if the user has taken the time out of their busy schedule to perhaps finish the registering they requested.

What we’re doing here will resemble what flyctl does, but we’re going to do it in Laravel.

The process is this:

A register command will:

Ask for a new CLI session (we get a token back)
Kick the user out to the browser to register, passing the session ID
Poll the API with the session ID to ask if the user finished registering

Our application needs:

API endpoints to handle creating a CLI session and checking on its status
Adjustments to the browser-based registration flow to capture the CLI session ID and associate it with a user who registered

Project Setup

We’ll create a new Laravel project, using Breeze to scaffold authentication. You can ➡️ view the repository ⬅️ for any details I hand-wave over.

Here’s a quick set of commands to scaffold everything out:

    # Create the web app that a user will interact with in the browser
composer create-project laravel/laravel browser-cli
cd browser-cli

composer require laravel/breeze --dev
php artisan breeze:install
npm i && npm run build

# Create a model, migration, and resource controller for
# CLI session handling
php artisan make:model --migration --controller --resource CliSession

# Create a CLI command that we'll use, but in reality would likely
# be a separate code base that a user gets installed to their machine
php artisan make:command --command register RegisterCommand

  

The model and migration are extremely vanilla. We get a UUID as a CLI session ID, and can associate it with a user.

Let’s dive into the details, starting with the register CLI command, since it’s the “top-level” thing we’ll be doing.

Register Command

I created an artisan register command within the same code base as the API. This is just easier for demonstration. In reality, you’d likely create a separate CLI-only app for users to install (and it would talk to your API, which is probably a whole separate Laravel code base).

The register command is going to get a new CLI session, kick the user to the browser, and then wait for registration to happen. It polls the API, asking if the user registered. When registration happens, the CLI will get a valid API token for the user related to the CLI session ID.

    <?php

namespace App\Console\Commands;

use Illuminate\Console\Command;
use Illuminate\Support\Facades\Http;
use Illuminate\Support\Facades\Process;

class RegisterCommand extends Command
{

    protected $signature = 'register';
    protected $description = 'Kick off user registration';

    public function handle()
    {
        // We're using the same code base for the API, so lets
        // just get the URL for cli-session stuff
        $baseUrl = url('/api/cli-session');

        // Get a new CLI session
        $cliSessionResponse = Http::post($baseUrl, [
            'name' => gethostname(),
        ]);

        if (! $cliSessionResponse->successful()) {
            $this->error("Could not start registration session");
            return Command::FAILURE;
        }

        $cliSession = $cliSessionResponse->json();

        $stop = now()->addMinutes(15);

        // TODO: Using "open" is different per OS
        Process::run("open ".$cliSession['url']);

        // Poll API for session status every few seconds
        $apiToken = null;
        while(now()->lte($stop)) {

            // check session status to see if user
            // has finished the registration process
            $response = Http::get($baseUrl.'/'.$cliSession['id']);

            // non-20x response is an unexpected error
            if (! $response->successful()) {
                $this->error('could not register');
                return Command::FAILURE;
            }

            // 200 response means user registered
            if ($response->ok()) {
                // response includes an API token
                $apiToken = $response->json('api_token');
                $this->info('Success! Retrieved API token: ' . $apiToken);
                break;
            }

            // Else I chose to assume we got an HTTP 202, 
            // meaning "keep trying every few seconds"
            sleep(2);
        }

        // TODO: Success! Store $apiToken somewhere for future use
        //       e.g. ~/.<my-app>/.config.yml 🦉
        $this->info('success!');
        return Command::SUCCESS;
    }
}

  

Just as advertised, we’re starting a session and then polling the API for its status (for up to 15 minutes).

Once the user registers, we get an API token back that we can use to perform future actions as that user.

One thing I skipped over is how we use the open command to open the web browser. How you do that changes per OS. See examples of how core Laravel does this across OSes here.

API Routes

We have /api/cli-session routes to handle! Within routes/api.php, we can register the resource controller. The interesting part is the CliSessionController itself:

    <?php

namespace App\Http\Controllers;

use App\Models\CliSession;
use Illuminate\Http\Request;
use Illuminate\Support\Str;

class CliSessionController extends Controller
{
    // Create a CLI session when requested by
    // the register command
    public function store(Request $request)
    {
        $request->validate([
            'name' => 'required',
        ]);

        $cliSession = CliSession::create([
            'name' => $request->name,
            'uuid' => Str::uuid()->toString(),
        ]);

        return response()->json([
            'id' => $cliSession->uuid,
            'url' => route('register', [
                'cli_session' => $cliSession->uuid
            ]),
        ]);
    }

    // Allow the register command to poll
    // for session status
    public function show(string $id)
    {
        $cliSession = CliSession::with('user')
            ->where('uuid', $id)
            ->firstOrFail();

        // Session exists but no user yet
        if (! $cliSession->user) {
            return response('', 202);
        }

        /**
         * User is associated with session 
         * (successful login or register).
         * Give them a new, usable API token
         */

        // Ensure no one can re-use this session
        $cliSession->delete();

        // TODO: Generate a for-real api token
        //       perhaps via Laravel Sanctum 🦉
        return [
            'api_token' => Str::random(32),
        ];
    }
}

  

It only has two jobs:

Let the register command start a session
Let the register command poll that session to see if the user finished registering

Creating a session just involves making a new cli_sessions table record with a new UUID. That UUID is passed back and becomes the session ID the register command uses to check on the session status.

The register command then polls the show method. When the user finishes registering, the it returns a valid API token for the App\Models\User associated with the CLI session. That’s the end of the process!

Browser Registration

We need to tweak the registration process so it becomes aware of the CLI session.

When a user registers, we need to associate the CLI session with the newly created user.

The API route checks for an associated user - if there is one, it assumes the user registered. It can then return an API key next time it is polled!

What those tweaks looks like depends if you’ve hand-rolled authentication, or used Breeze/Jetstream/Whatever. Here’s what it looks like for Breeze.

What we do is:

Include a cli_session hidden <input /> in the registration form with the UUID of the session (if present)
Adjust the registration POST request to find and update the CLI Session record to be associated with the just-created user

Relevant snippets from the adjusted Breeze-generated app/Http/Controllers/Auth/RegisteredUserController.php

    /**
 * Display the registration view.
 */
public function create(): View
{
    // 1️⃣ Add the CLI session UUID to the register view
    return view('auth.register', [
        'cli_session' => request('cli_session')
    ]);
}

/**
 * Handle an incoming registration request.
 *
 * @throws \Illuminate\Validation\ValidationException
 */
public function store(Request $request): RedirectResponse
{
    $request->validate([...]);

    $user = User::create([...]);

    event(new Registered($user));

    Auth::login($user);

    // 2️⃣ Associate user with cli session, if present
    if ($request->cli_session) {
        $cliSession = CliSession::where('uuid', $request->cli_session)
            ->firstOrFail();

        $cliSession->user_id = $user->getKey();
        $cliSession->save();

        // TODO: Create this route with message letting
        //       user know to head to back to CLI
        return redirect('/cli-session-success');
    }

    return redirect(RouteServiceProvider::HOME);
}

  

Two hand-wavy 👋 things:

You’ll need to update the auth.register view to add a hidden input and store the cli_session value
I didn’t bother creating a “success!” view that tells the user to head back to their terminal

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

That’s It

Overall, it’s not too bad - we didn’t need to overhaul our registration process! We just hooked into it a bit in a way that allowed for the addition of a CLI session.

The register command did most of the work, and all that entailed was asking for a UUID and polling an endpoint.

Other Wacky Ideas

The main blocker for CLI-browser communication is the communication itself. How could the browser “push” data to a terminal process?

In our case, we didn’t “push” data anywhere. The terminal command was polite - it asked for the CLI session, and then asked for the status of the CLI session.

Here’s 2 zany ideas, both of which sort of suck for PHP but might fit in with other languages with stronger concurrency/async.

Warning: Strong “draw the rest of the owl” vibes here.

Web sockets - Have your command and the browser connect to a web socket server, and communicate that way
Web server - Have your command start a web server, listening at localhost:<some-port>, then have your web app redirect to http://localhosts:<some-port>/success (or whatever) when registration is complete

Checkboxes, and Streamed updates with Livewire

2023-10-26T00:00:00+00:00

Need a place for your Laravel app in the cloud? Fly it with Fly.io, it’ll be up and running in minutes!

Imagine having a table with ten rows. Imagine even more, having a “Bulk Action” button above this table. When the user selects all ten rows from the table and clicks on the button, all ten rows are sent to the server for bulk processing.

Each row is processed one after the other, until finally, all rows complete and the server responds back to the client. This setup means that even though a portion of the group might’ve already completed its processing, this portion will not be updated as “processed” in the UI just yet—not until all other remaining parts of the group complete as well.

Well, what if, we want to progressively update the UI as each row completes? Do we create a new connection to poll for and reflect updates? Or do we create a web socket connection to listen for this change continuously?

Not quite! We don’t have to do any of the two above. Cause here comes Livewire’s wire:stream directive! Our ones-top directive for progressively listening and reflecting changes from the server.

Let’s get to it!

What We’ll Do

To showcase Livewire’s wire:stream directive, today, we’ll make use of a table displaying rows of “properties for sale”. Each row will have a corresponding checkbox a user can use to select different properties, and a button to send selected rows to the server for bulk processing.

Finally, we’ll make use of Livewire’s wire:stream directive to update our UI as each row completes processing, showing as each row goes through “WIP” and finally “Processed”—all this with only one request to the server.

Set Up

Start with creating a Livewire component with php artisan livewire:make my-table. This component’s view will contain a table, a bunch of rows, and a checkbox per row:

    <div>
    <table>
        <thead>
            <tr>
              <th></th>
              <th>Name</th>
              <th>Processed</th>
            </tr>
        </thead> 
        <tbody>
            <!-- List real estate properties -->
            @foreach( $properties as $prop )
            <tr>
                <!-- Yes Checkbox for each row! -->
                <td><input type="checkbox"></td>
                <td>{{$prop['name']}}</td>
                <td> TBD </td>
            </tr>
            @endforeach
        </tbody>
         {{ $properties->links() }}

  

It’ll also include a button to trigger a method called process() in the component class:

          <button wire:click="process">Process</button>
    </table>   
</div>

  

Then, in its class, we pass $properties containing row data to the view, and declare process() as a public method:

    public class MyTable
{
  // The method to process our rows
  public function process(){}

  public function render()
  {
      // Pass properties for sale data to the view
      return view('livewire.my-table',[
          'properties'=>
            Property::paginate(10)
      ]);
  }
}

  

And now we have our table, checkboxed rows, and Process button!

Wiring Checkboxes

Our table has a bunch of checkboxes, one for each row of data in the $properties array. Each checkbox represents its row’s ID, retrieved from $prop->id.

When the user clicks on the “Process” button, we want to send every selected checkbox’s ID to the server. So, in the component’s class, we’ll declare a public array public $ids = [];. Then, we’ll wire each checkbox to the array’s index at $prop->id—the checkbox’s ID:

We’ll use the row’s id, “$prop->id” to uniquely identify each checkbox.

    @foreach( $properties as $prop )
  <input type="checkbox"
+    wire:model="ids.{{$prop->id}}" >

  

With the setup above, if we were to select any checkbox, and do a dd( $this->ids ) in the process() method, we’ll get ids as an array of trues, with each index representing an ID of a checkbox checked by the user:

    array:2 [▼ // app/Livewire/MyTable.php
  11 => true // 11 and 12 as selected ids 
  12 => true
]

  

Streaming Updates with wire:stream

Now that we’re able to get the IDs of each row a user selects, it’s time to process these, and show in the UI as each row goes through the different statuses of processing: “WIP”, and finally, “Processed”.

First, we’ll have to associate a status attribute per item in the $ids array. At the current structure of this array, however, we can’t simply add a status attribute an item. We’ll have to first revise the array’s structure so it can hold an array of key-value pairs.

We can do this re-structuring by revising the wiring of each row:

    @foreach( $properties as $prop )
  <input type="checkbox"
+    wire:model="ids.{{$prop->id}}.checked" >

  

Notice we simply add a checked attribute. This will assign an array of key-value pairs to each row, the first item key being the checked key:

    // New structure!
array:2 [▼ // app/Livewire/MyTable.php:14
  11 => ['checked'=>true],
  12 => ['checked'=>true]
]

  

This structure will now allow us to add other attributes later on, just like the status attribute we’ve mentioned above.

Next, let’s check out how we can show progress as each row completes processing.

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Streaming with wire:stream

Livewire v3 now offers the incredible wire:stream directive. We can “stream” progress as each row completes processing.

In fact, we can even stream a status even before we start processing an item. We can say a row is WIP by updating $this->ids[]['status'] to “WIP”, then “streaming” this update to the client:

to represents the wire:stream block we want to stream the update to,

content represents the public attribute we want to show the update of,

and replace tells Livewire that we want to replace the previous content

    foreach( $this->ids as $id=>$idDetails ){

     // Update status to WIP
    $this->ids[ $id ][ 'status'] = 'WIP';

    // Stream this change to the client
    $this->stream(
        to: 'process-stream'.$id,
        content:  $this->ids[ $id ][ 'status'],
        replace: true 
    );

    // Processing might take 3 seconds...
    sleep( 3 ); 
}

  

This stream can now be received in the view by setting up a matching receiver:

    @foreach( $properties as $prop )
  <tr>
      <td>
+     <td wire:stream="process-stream.{{ $prop->id }}">
+        {{ $content }} 
      </td>
  </tr>
  @endforeach

  

Then, once an item completes processing, we can update the status attribute to “Processed” and finally stream this change to the UI again:

    foreach( $this->ids as $id=>$idDetails ){
    // Before processing stream here...

    // Processing takes up to 3 seconds
    sleep(3);

    // Completed update and stream
    $this->ids[ $id ][ 'status' ] = 'Processed!';

+    $this->stream(
+      to: 'process-stream'.$id,
+      content: $this->ids[$id]['status'],
+      replace: true
+    );
}

  

And that’s it! We now have a bunch of selectable rows, we can send them to the server for processing, and we get to update each row in the client table, real time!

Livewire v3: Modelable, Events, and Data Sharing

2023-09-27T00:00:00+00:00

Need a place for your Laravel app in the cloud? Fly it with Fly.io, it’ll be up and running in minutes!

Oftentimes, we use drop-downs in combination with other html elements to make our website interactive.

One kind of interaction is inter-dependency amongst several drop-downs. This happens when changing a selected option of one drop-down affects the choices given for another drop-down. In our last article, we created dynamic, inter-dependent drop-downs. These drop-downs were dynamically instantiable as they all used one Livewire component as template( named filter component ). Then, to allow inter-dependency among the drop-downs, we created another component called filter-section to manage data and data-dependency for each filter drop-down instance.

Another kind of interaction involves the drop-downs affecting other html elements, like say, filtering data on a table.

Of course, given the nature of dynamically instantiated elements, we need to be able to uniquely identify and track each drop-down instantiated. Furthermore, if our table element is found in a separate Livewire component, we need to find a way to share the data from each filter drop-down to the table component.

Today, we’ll get around this data sharing easily with Livewire Modelable and client-side dispatched events.

—Let’s get to it!

Dynamic Drop-Down Set Up

Let’s say we use a filter-section component to manage a bunch of drop-downs. It would have a public attribute $filters that contains an array of data which we’ll use to initialize a bunch of drop-downs:

    <!-- resources/livewire/filter-section.blade.php -->
<div>
    <button>Apply</button>
    @foreach( $filters as $key=> $filter )
        <livewire:filter
          :key="$key.now()"
          :label="$filter['label']"
          :options="$filter['options']"
        />
    @endforeach     
</div>

  

Notice however that each item initializes another Livewire component called filter. This component contains the template we use to create our drop-downs:

    <!-- resources/views/livewire/filter.blade.php -->
<div>
  <!-- Show the label of the filter -->
  <label>{{ $label }}</label>

  <!-- Filter -->
  <select wire:model="value">
    @foreach( $options as $option )
      <option value="{{ $option['id'] }}">{{ $option['name'] }}</option>
    @endforeach
  </select>
</div>

  

Let’s say that the $filters array contains three drop-down data: “Offer Type”, “Property Type”, and “Sub Property Type”. We should get a view like so:

We’re going to use these drop-downs to show filtered results in a table. The first thing we need to do, then, is to get the selected option of each filter drop-down.

How exactly do we do that?

Uniquely Identifying Drop-Downs

In our setup, we have a parent component filter-section, and it dynamically initializes a bunch of child filter components through an array of data. Each filter component instance contains a drop-down element.

If we want to get the selected option for each drop-down, we’ll have to of course, uniquely identify each one. So, first, make sure that the $filters array includes a unique key value for each drop-down data. Then, we’ll need a ‘value’ attribute that will represent the option selected for each drop-down:

    public function mount()
{
  $this->filters = [
+       'offer_type'=>[     // Unique key
            'label'=> 'OFFER TYPE',
            'options'=> OfferType::all()->toArray(),
+           'value' => 0    // Value attr
        ],
+        'property_type'=>[  // Unique key
            'label'=>'PROPERTY TYPE',
            'options'=> PropertyType::all()->toArray(),
+           'value' => 0  // Value attr
        ],
+        'sub_property_type'=>[ // Unique key
            'label'=>'SUB PROPERTY TYPE',
            'options'=> SubPropertyType::all()->toArray(),
+           'value'=>0   // Value attr
        ]
  ];
}

  

With this setup, we can refer to each drop-down’s selected value through the attribute $filters.[drop_down_key].value.

Now that we have an attribute name to identify each drop-down value, the next step is to link each attribute to its respective drop-down element in our view. In Livewire, we link server attributes with elements in the blade through the use of wire:model directives:

    <!-- resources/livewire/filter-section.blade.php -->

@foreach( $filters as $key=> $filter )
  <livewire:filter
    ...
+   :wire:model="'filters.'.$key.'.value'"
  />
@endforeach     

  

Usually, that’s how we would link attributes to html elements. But notice that filter-section only has direct access to its filter child instances and not the drop-down elements we want to wire to.

This syntax then won’t work alone. Because, after all, how would Livewire know which element in the filter component’s blade should it wire an attribute with?

Enter Modelable: Linking Parent and Child Attributes

Livewire provides us with its Modelable attribute feature. It allows us to match a parent wire:model with any of its child components’ attributes, and ultimately, that child component’s html element.

With it, we can have attributes in our parent component in sync with child components’ attributes!

Applying this directive is just a two-step process. The first step we’ve already done above, which is wiring a specific filter-section‘s attribute to the instance of its filter child component.

Next, we need to let the filter component know where to map the wired model it received. We can do this by attaching “Modelable” to the attribute we want to link with. Since our filter component has $value wired to its select element, we want to specifically attach to this attribute:

    /* app/Livewire/Filter.php */

+  use Livewire\Attributes\Modelable;

class Filter extends Component
{
+  #[Modelable] 
   public $value = '';

  

And that’s it! Now that we’ve attached Modelable to $value and wired it to the parent’s $filters.[drop_down_key].value attribute, any changes to $value would also be reflected in its corresponding $filters.[drop_down_key].value, thereby giving filter-section access to the drop-down values found in each instantiation of filter child components.

Neat, right?

Let’s test that out with a little bit of Alpine and console logging. From our parent component, filter-section, we have an Apply button. When the user clicks on this button, we can log the value of a specific drop-down:

    <button
  x-on:click="console.log( $wire.filters.sub_property_type.value )"
>

  

And…tada! Clicking on the Apply button shows us the value of our intended drop-down!

Now that we have a way to get each selected drop-down data, let’s finish this by sharing them to another component — our table component.

The table component will make use of Livewire’s WithPagination trait to display rows of our data. And to allow filtering, we’ll also add in a public attribute $filterList which we’ll pass to the pagination query:

    /* app/Livewire/Table.php */
use Livewire\WithPagination;

class Table extends Component
{
  use WithPagination;

  public $filters = [];

  public function render()
  {
      return view('livewire.property-table',[
          'rows'=>Property::filter( $this->filters )->paginate(10)
      ]);
  }
}

  

Then, with the data passed through the $rows variable in render(), we create the view which includes the data and pagination links:

    <!-- resources/views/livewire/table.blade.php -->
<table>
    <thead>
        <tr>
            <th class="bg-blue-100 border text-left px-8 py-4">Name</th>
            <th class="bg-blue-100 border text-left px-8 py-4">Price</th>
        </tr>
    </thead> 
    <tbody>
        @foreach( $rows as $row )
          <tr>
              <td>{{$row['name']}}</td>
              <td>{{$row['price']}}</td>
          </tr>
        @endforeach
    </tbody>
    {{ $rows->links() }}
</table>

  

Now if we declare this table component in our main page, we should finally get a complete setup like so:

The final question in this article we’ll be answering is, how do we pass data from one component, to another?

Client-Dispatched Events

To share data from one component to another, we usually dispatch events in Livewire. Livewire components can then listen to these events and react accordingly.

To immediately and thriftily dispatch an event, we’ll dispatch one in our blade template.

When the user clicks on the Apply button in our filter-section component, we want to take the selected drop-downs data it has access to, and share these with the table component.

To do so, we can dispatch an event containing each drop-down’s selected value from a button click:

    <!-- resources/views/livewire/filter-section.blade.php -->

<button 
  wire:click="$dispatch('apply-filter', { 
    offer_type_id: $wire.filters.offer_type.value,
    property_type_id: $wire.filters.property_type.value,
    sub_property_type_id: $wire.filters.sub_property_type.value,
  })"
>Apply</button>

  

Since this apply-filter event is available on our entire page, we can listen to this from our table component’s view( or any other component really! ) and react.

We can take the selected drop-down data from event.detail, and directly update the $filtersList variable in our table component with it:

    <script>
window.addEventListener('apply-filter', event => {
    console.log( 'received event from apply filter in table!', event.detail );
+   @this.set( 'filtersList', event.detail );
});
</script>

  

And that’s it! We’ve shared our selected drop-down values with the table component!

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Once Livewire updates the $filtersList attribute in the server using @this.set(), Livewire would afterward call the render() method of table component due to an update to its public attribute.

This time, the $filtersList will contain data the user selected, and will now be used to filter the rows for our table!

Re-usable, Dependent Dropdowns in Livewire v3

2023-09-13T00:00:00+00:00

In just a few steps, deploy your Laravel application on Fly.io, you’ll be up and running in minutes!

How do we create dependency among dropdowns that use the same, re-usable component? One way we can solve this is by utilizing Livewire’s powerful component nesting capabilities.

Let’s check it out, here’s a repository for reference.

Problem

We have a re-usable filter component that contains a select dropdown:

    <!-- resources/views/livewire/filter.blade.php -->
<div>
  <!-- Show the label of the filter -->
  <label>{{ $label }}</label>

  <!-- Filter -->
  <select wire:model="value">
    @foreach( $options as $option )
      <option value="{{ $option['id'] }}">{{ $option['name'] }}</option>
    @endforeach
  </select>
</div>

  

We want to use this to create 3 different dropdown filters ( Offer Type, Property Type, and Amenities ) for our page.

One key point is that changes on the Property Type dropdown should filter the options in the Amenities dropdown.

How do we create such dependency that come from the same component?

Solution: A Parent Data Manager

At the end of the day, what we really have here is data dependency: Data from one filter component reacting to data from another filter component.

And in the beginning, where did we even get data to instantiate these two components? —Somewhere outside of it.

So, if we get data from outside of the component, why not handle data stuff like dependency outside of the component as well?

We can create a filter-section component that will manage the dropdowns. It will be responsible for managing data, rendering, and coordinating events across its filter child components.

And, thanks to Livewire v3’s component nesting capabilities, all these data and event coordination, as well as re-rendering of UI changes, is going to be smooth.

Set Up

First, create a filter component, it’s the “template” to our dropdowns. Afterwards, create another component called filter-section, this will act as the parent manager.

Getting Data

The filter-section will have a public attribute, $filters, that will contain data on the three dropdowns we want to render in our page:

    /* app/Livewire/FilterSection.php */

public $filters = [];

public function mount()
{
  $anyArray = ['id'=>0,'name'=>'Any'];
  $this->filters = [
       'offer_type'=>[
            'value' => 0,
            'label'=> 'OFFER TYPE',
            'options'=> [ $anyArray ]+OfferType::all()->toArray()
        ],
        'property_type'=>[
            'value' => 0,
            'label'=>'PROPERTY TYPE',
            'options'=>[ $anyArray ]+PropertyType::all()->toArray()
        ],
        'amenity'=>[
            'value' => 0,
            'label'=>'AMENITIES',
            'options'=>[ $anyArray ]+Amenity::all()->toArray()
        ]
  ];
}

  

Initializing Child Components

In its view, we’ll instantiate a bunch of child filter components using the items in $filters. For each item, we initialize a filter component with the item’s $label, $options, and $value:

    <!-- resources/livewire/filter-section.blade.php -->
<div>
    @foreach( $filters as $key=> $filter )
        <livewire:filters
          :key="$key.now()"
          :label="$filter['label']"
          :options="$filter['options']"
          :value="$filter['value']"
        />
    @endforeach     
</div>

  

Saving the code snippet above should give us an instance of filter component for each item in the $filters array:

Next, let’s focus on the :key attribute declared above.

The Importance in `:key`

Including a :key identifier in each filter child component is a vital, mandatory requirement. Livewire uses this :key attribute to keep track of children components during re-rendering of changes in the page.

And, speaking of “re-rendering”, Livewire by default only renders a child component when the child component has not yet been rendered before.

In fact, a parent component maintains this list called “children”. It contains the :key of all child components previously rendered. Whenever the parent needs to re-render, it skips any child whose key has already been recorded in the list.

Since our “Amenities” dropdown has already been rendered in initial page load, Livewire’s not going to re-render it afterwards. But see, we need re-rendering for changes to reflect, so what do we do?

This, is where $key.now() comes in. This should generate a unique key for our dropdown every time, allowing it to have a different key from before, forcing the re-rendering of the component—and reflecting new changes to our dropdowns ✨

Dependent Filters

Now, let’s go over to the “dependency” part.

When a user selects a “Property Type” option, we want to filter the “Amenities” options to only include options for that specific property type.

There’re two parts to this: Listening for “Property Type” changes, and Requesting the parent component to filter its “Amenities” dropdown. Add this logic to the filter view:

    <!-- resources/views/livewire/filter.blade.php -->
<select 
     x-on:change="
        $wire.label == 'PROPERTY TYPE' && 
        $wire.$parent.filterAmenity( $wire.value )"
>

  

What’s happening above?

First, notice we use $wire everywhere. It’s this magical JavaScript object that represents the current component that caused the action.

Then, for the bigger picture we make use of some Alpine. We combine a quick x:on-change to listen for option changes, and add in a conditional statement to specifically listen to a component that has $wire.label “PROPERTY TYPE”.

Finally, we proceed with calling the parent’s method filterAmenity() to filter the data for the “Amenities” dropdown using the argument passed.

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Filtering with PropertyTypeId

Lets wrap this up by declaring the parent’s filterAmenity() method called from above. We declare it as a public method in our filter-section component’s class:

    /* app/Livewire/FilterSection.php */

public function filterAmenity( $propertyTypeId )
{
    $this->filters['amenity'] = [
        'value'=>0,
        'label'=>'AMENITIES',
        'options'=>Amenity::filterPropertyType( $propertyTypeId )
        ->get()->toArray()
    ];
}

  

The function simply receives a $propertyTypeId, and uses it to update the options of the amenities filter item. Save the changes, and see our dropdowns in action!

The Takeaway

When introducing dependency across dropdowns that use the same, re-usable component, there are three general questions that need to be answered:

How can we listen to change from a specific dropdown?
How can we communicate this change to another specific dropdown?
How can this change be reflected in the UI?

The solution we used today relied on a data manager component to communicate dependency among our dropdowns. And with Livewire, we were able to easily implement this solution.

We combined Livewire’s in built Alpine x-on:change, and $wire object to listen for changes from a specific dropdown ( answering Q#1 ). Finally, we accessed the $parent object from our child component to call its parent’s method to filter the data for the dependent “Amenities” dropdown( answering Q#2! ).

With Livewire, we didn’t even have to worry about the third question! We just had to provide the proper always-unique :key value and Livewire handled everything for our Amenities dropdown to reflect its brand new, filtered options.

All this in a few lines of code. Amazing

Ad Hoc Tasks, GitHub Actions Style

2023-09-04T00:00:00+00:00

What if you could spin up a VM, do a task, and tear it all down super easily? That’d be neat, right? Let’s see how to spin up Laravel application on Fly.io and run some ad-hoc tasks!

We’re going to create a setup where we can instruct Fly.io to spin up a VM and run some code. The code will read an instructions.yaml file and follow its instructions. The neat part: We can create the YAML file on-the-fly when we spin up the VM.

This lets us create one standard VM that can do just about any work we need. I structured the YAML a bit like GitHub Actions.

In fact, I built a similar thing before:

I re-created github actions on @flydotio machines in this little proof of concept

👨‍🍳 Make a yaml, sprinkle in some go, make an API request to Fly, and away we go. pic.twitter.com/HjfF6k6ohX
— Chris Fidao (@fideloper) August 11, 2023

It uses Golang, but here we’ll use Laravel 🐐.

Here’s a repository with the code discussed.

The Code

The code is pretty straight forward (partially because I ask you to draw the rest of the owl). Within a Laravel app, we’ll create a console command that does the work we want.

Here’s what I did to spin up a new project and add a console command to it:

    composer create-project laravel/laravel ad-hoc-yaml
cd ad-hoc-yaml

composer require symfony/yaml

php artisan make:command --command="ad-hoc" AdHocComputeCommand

We need to parse some YAML, so I also included the symfony/yaml package.

And the command itself is quite simple 🦉:

    <?php

namespace App\Console\Commands;

use Symfony\Component\Yaml\Yaml;
use Illuminate\Console\Command;
use Illuminate\Support\Facades\Log;

class AdHocComputeCommand extends Command
{
    protected $signature = 'ad-hoc';

    protected $description = 'Blindly follow some random YAML';

    public function handle()
    {
        try {
            $instructions = Yaml::parseFile("/opt/instructions.yaml");
        } catch(\Exception $e) {
            Log::error($e);
            $this->error($e->getMessage());
            return SELF::FAILURE;
        }


        foreach($instructions['steps'] as $step) {
            // draw the rest of the <bleep>ing owl
        }

        return SELF::SUCCESS;
    }
}

  

For a slightly-more-fleshed out version of this code, see this command class here.

Package it Up

Fly uses Docker images to create (real, non-container) VMs. Let’s package our app into a Docker image that we can deploy to Fly.io.

We’ll borrow the base image Fly.io uses to run Laravel apps (fideloper/fly-laravel:${PHP_VERSION}). We’ll just assume PHP 8.2:

    FROM fideloper/fly-laravel:8.2

COPY . /var/www/html

RUN composer install --optimize-autoloader --no-dev

CMD ["php", "/var/www/html/artisan", "ad-hoc"]

Once that’s created, we can build that Docker image and push it up to Fly.io. One thing to know here: You can only push images to Fly.io’s registry for a specific app. The image name to use must correspond to an existing app, e.g. docker push registry.fly.io/some-app:latest. You can use any tags (e.g. latest or 1.0) as well as push multiple tags.

So, to push an image up to Fly.io, we’ll first create an app (an app is just a thing that houses VMs) and authenticate Docker against the Fly Registry. Then, when we spin up a new VM, we’ll use that image that’s already in the registry.

This is different from using fly deploy... which builds an image during deployment, and is meant more for hosting your web application. Here we’re more using Fly.io for specific tasks rather than hosting a whole application.

The following shows creating an app, building the Docker image, and pushing it up to the Fly Registry:

    APP_NAME="my-adhoc-puter"

# Create an app
fly apps create $APP_NAME

# Build the docker image
docker build \
    -t registry.fly.io/$APP_NAME \
    .

# Authenticate with the Fly Registry
fly auth docker

# Push the docker image to the Fly Registry
# so we can use it when creating a new VM
docker push registry.fly.io/$APP_NAME

  

This article is great at explaining the fun things you can do with the Fly Registry.

We make 2 assumptions here:

You have Docker locally
You’re on an Intel-based CPU

Pro tip: If you’re on a ARM based machine (M1/M2 Macs), you can actually VPN into your Fly.io private network and use your Docker builder (all accounts have a Docker builder, used for deploys) via DOCKER_HOST=<ipv6 of builder machine> docker build ....

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Run It

To run our machine, all we need is a YAML file and to make an API call to Fly.io.

As mentioned before, Machines (VMs) spun up via API call let you create files on-the-fly! What you do is provide the file name and the base64'ed file contents. The file will be created on the Machine VM before it runs your stuff.

Here’s what the code to make such an API request would look like within PHP / Laravel:

    # Some YAML, similar to GitHub Actions
$rawYaml = '
name: "Test Run"

steps:
  - name: "Print JSON Payload"
    uses: hookflow/print-payload
    with:
      path: /opt/payload.json

  - name: "Print current directory"
    run: "ls -lah $(pwd)"

  - run: "echo foo"

  - uses: hookflow/s3
    with:
      src: "/opt/payload.json"
      bucket: "some-bucket"
      key: "payload.json"
      dry_run: true
    env:
      AWS_ACCESS_KEY_ID: "abckey"
      AWS_SECRET_ACCESS_KEY: "xyzkey"
      AWS_REGION: "us-east-2"
';

$encodedYaml = base64_encode($rawYaml);

# Some random JSON payload that our YAML
# above references
$somePayload = '
{
    "data": {
        "event": "foo-happened",
        "customer": "cs_1234",
        "amount": 1234.56,
        "note": "we in it now!"
    },
    "pages": 1,
    "links": {"next_page": "https://next-page.com/foo", "prev_page": "https://next-page.com/actually-previous-page"}
}
';

$encodedPayload = base64_encode($somePayload);

# Create the payload for our API call te Fly Machines API
$appName = 'my-adhoc-puter';
$requestPayload = json_decode(sprintf('{
    "region": "bos",
    "config": {
        "image": "registry.fly.io/%s:latest",
        "guest": {"cpus": 2, "memory_mb": 2048,"cpu_kind": "shared"},
        "auto_destroy": true,
        "processes": [
            {"cmd": ["php", "/var/www/html/artisan", "ad-hoc"]}
        ],
        "files": [
            {
                "guest_path": "/opt/payload.json",
                "raw_value": "%s"
            },
            {
                "guest_path": "/opt/instructions.yaml",
                "raw_value": "%s"
            }
        ]
    }
}
', $appName, $encodedPayload, $encodedYaml));

// todo 🦉: create config/fly.php 
// and set token to ENV('FLY_TOKEN');
$flyAuthToken = config('fly.token');

use Illuminate\Support\Facades\Http;

Http::asJson()
    ->acceptJson()
    ->withToken($flyAuthToken)
    ->post(
        "https://api.machines.dev/v1/apps/${appName}/machines", 
        $requestPayload
    );

  

I created an artisan command that does that work here.

In your case, you might want to trigger this in your own code whenever you want some work to be done.

After you run some tasks, you should see the Machine VM spin up and do its work! Two things to make this more fun:

Liberally use Log::info() in your code so Fly’s logs can capture what’s going on (helpful for debugging)
Set your Logger to use the stderr logger so Fly’s logging mechanism can get the log output

Assuming that’s setup, you can then run fly logs -a <app-name> to see the log output as the Machine VM boots up, runs your code, and then stops.

SPA NavBars and Highlighting, Livewire v3 Style!

2023-08-29T00:00:00+00:00

In just a few steps, deploy your Laravel application on Fly.io, you’ll be up and running in minutes!

Livewire’s new wire:navigate directive is brilliant. It delivers the combined forces of SEO-friendly rendering, and the smoothness of an SPA powered user experience.

Today, we’ll create an SPA powered NavBar to provide seamless page navigation across nav links, and along the way, answer a very important question: how do we highlight the active link in an SPA powered nav bar?

Let’s get to it!

Let’s say, we have a Laravel Livewire app. And in it, three full page components:

    /* routes/web.php */
# Bind routes to Livewire components to create Full Page goodness
Route::get('/', \App\Livewire\HomePage::class );
Route::get('/about', \App\Livewire\AboutPage::class );
Route::get('/contact', \App\Livewire\ContactPage::class );

  

All three of these components share the same layout file which contains a Livewire component called “nav-bar” attached on top of the page:

    <!-- resources/views/app.blade.php -->
<html>
<head>...</head>
<body>
+   <livewire:nav-bar />
    {{ $slot }}
</body>
</html>

  

Thanks to this nav-bar, users get a “global” space for navigating among the three pages:

    <!-- resources\views\livewire\nav-bar.blade.php -->
<nav id="dir-nav">
  <a href="/">Home</a>
  <a href="about">About</a>
  <a href="contact">Contact</a>
</nav

  

This nav-bar is well and good, but it still has room for improvement. See, moving between its links still take full-page reloads. Full page reloads disrupt the flow of user interaction— users have to wait for an entire new page to finish loading before new content can be displayed. It’s not the worst experience, but well, we can still definitely improve it.

Enter `wire:navigate`

Livewire v3 just recently (officially) released! With it comes the revolutionary wire:navigate directive. It allows an “SPA” experience by loading the requested page in the background, and updating parts of the current page to reflect new content.

Simply attach the directive to any link:

      <a href="..." wire:navigate>...</a>

  

And behold—A seamless navigation experience for our users right at the fingertips of a single directive:

—we get to decide when to do full page reloads OR background content loading.

—on top of that, Livewire renders initial component content with the page: this means each of our pages’ initial content is available for friendly SEO processing!

“Adaptable” attach on the go, SEO friendly SPA—that’s wire:navigate—Livewire’s SPA on a silver platter 😎

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Highlighting the Active Link

Alright, now we have an SPA powered experience for our nav-bar. This fact leads us to a very important question:

How do we highlight the active link in an SPA powered navbar? When a user clicks on a link and visits a page, we want to highlight that recently clicked link as “active”. To go about this highlighting, we’ll have to make changes in our nav-bar: we highlight the <a> tag that matches the current page’s identifier $page:

    <!-- resources/views/livewire/nav-bar.blade.php -->

<!-- Highlight a link if page's identifier matches its value -->
<a href="..." wire:navigate 
    @if($page=='...') class="underline" @endif
>...</a>

  

Now the question is, how exactly do we get the value that identifies the current $page that’s active? Let’s look at three ways:

1. New Page Title

One value that can identify the active link is the current page’s title. Livewire conveniently gives an easy way to set a full component’s title through its [#Title] attribute.

Simply include a [#Title] attribute in every full-page components’ render function:

    /* App\Livewire\HomePage.php */
#[Title('Home Page')]
public function render(){...}

/* App\Livewire\AboutPage.php */
#[Title('About Page')]
public function render(){...}

/* App\Livewire\ContactPage.php */
#[Title('Contact Page')]
public function render(){...}

  

And we’ll automatically receive a page’s title in the layout file through a $title attribute. We can then pass $title to the nav-bar component, and fill in our $page value:

    <!-- resources/views/app.blade.php -->
<html>
<head><title>{{ $title ?? 'Page Title' }}</title></head>
<body>
+   <livewire:nav-bar :page="$title"/>

  

2. New Page URL

A second way to get the active link is through the page’s URL. From the NavBar component, get the page’s URL through url()->current(). Then parse the route from this URL string, and assign it to $page.

We set this assignment inside the render() method so that the URL is refreshed every time the nav-bar is rendered, which happens every time we navigate to a new page:

    /* App\Livewire\NavBar.php */
public function render(){
  // $page comes from parsing the route from the url
  $this->page = $this->getPage( url()->current() );
  return view('livewire.nav-bar');
}

public function getPage( $url ){ 
  // Get page from url
  $url = parse_url($url);
  if( isset($url['path']) )
      return trim( $url['path'] ,'/' );
  else    
      return '/';
}

  

3. `window.location`, and Alpine

Is there a way we can update $page and highlight our nav-link from the browser side of things? window.location.pathname can identify the current page’s route from the frontend, we’ll use this to update the value of $page client side.

To update $page, we might try to use vanilla JavaScript. The official docs page advise running JavaScript inside its hook livewire:navigated to run the scripts on page visit.

At the time of writing however, the caller of this hook is called before the new page’s url is pushed into window.history. This is why the route we receive inside a listener for livewire:navigated gives the previous route instead of the new one:

    /* This is not going to work for our use case: */
document.addEventListener('livewire:navigated',()=>{
    /* This will log the old route, not the current one */
    console.log('New url:',window.location.pathname);
});

  

Instead of relying on plain JavaScript which we need to place in the right point of code, we can instead use Livewire v3’s built-in AlpineJS to directly update the value of $page.

See, Alpine provides us a “shortcut” syntax for doing JavaScript things, furthermore, it allows us to easily do our intended action at just the right point of time:

    <!-- resources/views/livewire/nav-bar.blade.php -->
<nav
  x-init="$wire.set('page',window.location.pathname)"
>

  

x-init is a way to hook into the initialization phase of an Alpine element( which Livewire components are! ). Since Alpine is initialized after transition to the new page, calling window.location during x-init gives us the updated, new route!

What did we Accomplish?

Along our nav-bar journey, we learned about the use of the magically passed [#Title] attribute. We also learned that updates done in the render method are reflected even on wire:navigate-ed pages. Finally, we learned a much quicker way to do updates client-side after “wire:navigated” by relying on Alpine’s directives and update syntax.

And here we are, six minutes long: we now have an SPA powered( SEO friendly ), “active link” highlighting, NavBar—Livewire v3, wire:navigate style!

A NativePHP Example

2023-08-28T00:00:00+00:00

Fly.io can run Laravel apps globally. It’s a great place to host an API - perhaps for your Native apps! Deploy your Laravel API on Fly.io, you’ll be up and running in minutes!

NativePHP is out (in alpha currently), and I’ve had a great time building a desktop app in Laravel.

Along the way I’ve discovered a few not-yet-documented goodies that I’m excited to share.

Writing Laravel and seeing it translated into “native” desktop things (like a menu) feels powerful in a way that’s hard to describe.

What I like about desktop apps is that a lot of them need an API to be useful. Running an API (globally!) is what Fly.io is best at! So it seemed appropriate to check out NativePHP and see what I could come up with.

In this case, I’m going to talk to Fly.io’s API since it’s an example of a global API.

We’re going to make a menu-only application that lists out your Fly.io applications and their status.

To see the final version of the code, head here.

Setting Up

As per usual, I created a new Laravel application to kick this off. Then I followed the installation instructions for NativePHP:

    composer create-project --prefer-dist laravel/laravel native-fly
cd native-fly

composer require nativephp/electron

# I had to do this as NVM wasn't
# yet initialized for me
nvm use 16

php artisan native:install
php artisan native:serve

  

NativePHP is still in Alpha. I had an error about a missing native:config Artisan command each time I started the app. If you see that, it’s fine, you can ignore it.

We have a barebones app! It opens Laravel’s default route (/, aka the welcome route), and renders the HTML/CSS/JS for us in an application window (that window should pop up for you automatically).

Keep in mind that user interactions will be powered by Javascript. The way to think about this is that PHP is still the application code (“server”), but the part the user interacts with is still html/css/javascript. Servery things - like making API calls or talking to a local SQLite database - is happening in PHP.

NativePHP is actually running a web server, so you can act as if the standard server-client thing is happening. Pretend the application windows (and menus) are a browser.

The main bits of configuration and setup of your application occur when your application first boots up. As a result, a lot of our application’s configuration will happen in the NativeAppServiceProvider class.

In our case, we want a menu app, so we’ll get rid of the call to open an application window by default, and then create a Menu:

    # File app/Providers/NativeAppServiceProvider.php

public function boot(): void
{
    // Window::open();
    MenuBar::create();
}

  

The default NativePHP icon is used in the menu bar! Let’s update it to something more Fly-ish.

We’ll grab the Fly.io brand mark, resize it to 22w (default size) and 44w (for Retina displays), and save them at storage/app/flyTemplate.png and storage/app/flyTemplate@2x.png.

Then we can update our NativeAppServiceProvider again to set the icon():

    # File app/Providers/NativeAppServiceProvider.php

public function boot(): void
{
    // Window::open();
    MenuBar::create()
        ->icon(storage_path('app/flyTemplate.png'));
}

  

With the usage of ...Template.png, MacOS will grab the @2x version on Retina screens automatically.

It looks…good enough for our use case 😅. MacOS does some things to convert it to monochrome, and adjusts for light vs dark mode.

By default, the app displays the default view (Laravel’s default page on our fresh install). The menu does the same! In fact, if you click a link, it behaves like a browser - going to the next web page within the menu. Sorta neat, but not what we want.

I’m really enjoying that this is basically a little browser as a menu 😂

You can click links! I ended up with Laravel’s docs in my menu. pic.twitter.com/kKpE9ijeOT
— Chris Fidao (@fideloper) August 21, 2023

Instead, let’s make our own layout and get a view going. We’ll use that (and a new route) for our menu.

I followed the Laravel guide on Tailwind’s site to get basic Tailwind in place. Then I created a layout file, and a view named apps, which we’ll work out of.

    <!-- File resources/views/components/layout.blade.php -->
<!doctype html>
<html>
<head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    @vite(['resources/css/app.css', 'resources/js/app.js'])
</head>
<!-- 
    note the use of bg-transparent
    we need that for later...
-->
<body class="font-sans antialiased bg-transparent">
{{ $slot }}
</body>
</html>

  

And then our view file:

    <!-- File resources/views/apps.blade.php -->
<x-layout>
    <div class="p-6">
        Stuff Here
    </div>
</x-layout>

  

Then we need to create a route to serve the menu stuff up:

    # File routes/web.php
Route::get('/apps', \App\Http\Controllers\AppsController::class)
    ->name('apps');

  

That’s served from an invokable controller where we just just return our apps view:

    # File app/Http/Controllers/AppsController.php
public function __invoke(Request $request)
{
    return view('apps');
}

  

Finally, update the provider NativeAppServiceProvider to tell the menu to use our new route to render the menu:

    # File app/Providers/NativeAppServiceProvider.php
public function boot(): void
{
    // Window::open();
    MenuBar::create()
        ->icon(storage_path('app/flyTemplate.png'))
        ->route('apps');
}

  

Pretty basic, but we’re getting somewhere!

Wiring the View File

We’re going to style a bunch of stuff.

First, the background of our menu is white. Gross! Let’s copy the default MacOS style, which is an opaque style that blurs the stuff behind it. The lingo for this is “vibrancy”, and there’s a NativePHP helper for it.

Remember how I pointed out the <body> tag’s use of bg-transparent? That makes this work. We can use the blendBackgroundBehindWindow() method to our menu and make it look “native”:

    # File app/Providers/NativeAppServiceProvider.php
public function boot(): void
{
    // Window::open();
    MenuBar::create()
        ->icon(storage_path('app/flyTemplate.png'))
        ->route('apps')
        ->blendBackgroundBehindWindow();
}

  

And our menu becomes opaque. Or whatever MacOS marketing would call it - “vibrant” in their case, although I would have gone with “delicious”

Next, we’re going to make a list of Apps running in our Fly.io account. I stole styles from the Fly.io console you see at https://fly.io/dashboard. Here’s a selection of HTML:

    <!-- File resources/views/components/apps.blade.php -->
<x-layout>
    <div>
        <ul class="...">
            <li class="...">
                <!-- fancy HTML here to display the things -->
                <!-- see https://github.com/fideloper/nativephp-fly -->
            </li>
        </ul>
    </div>
</x-layout>

  

You can get a glimpse of the full <li> section here. It’s the “final” version we end up with by the end of the article.

Let’s wire up the actual Fly.io API data to that and see what it looks like!

Fly API Call

When we first render our menu, we want to call the Fly API to get information about our running apps, within a given org.

This requires some boilerplate!

We need some config for Fly.io’s API token and organization.
We make API calls to Fly’s GraphQL API so we can get specific information we want (similar to what the Fly Dashboard does behind the scenes)
We list the apps and wire up our view!

Config

In a real-world desktop app, you’d want this to be something a user can set in a setup page when they first open the app.

In my case, I created a config/fly.php file and threw the needed config into Laravel’s .env file.

    # file config/fly.php
<?php

return [
  'token' => env('FLY_API_TOKEN'),
  'org' => env('FLY_ORG_NAME', 'personal'),
];

  

And we set the env vars:

# File .env
FLY_API_TOKEN=SOME_TOKEN_HERE
FLY_ORG_NAME=personal

API Calls

I created a class that makes API calls to Fly.io’s Graphql api. This is used by Fly’s tooling (e.g. flyctl) but it’s publicly available for you to play with.

This class is boring, and I’m not pasting it here. But it might be interesting to see example GraphQL queries you can make! Check it out here.

Listing Apps

Finally, we can wire up our view file to list the results of our API calls

    # file app/Http/Controllers/AppsController.php
public function __invoke(Request $request)
{
    return view('apps', [
        'apps' => (new Fly)->getApps(),
    ]);
}

  

If you try that, it should (hopefully) work!

Pretty neat! We did a simple app!

Linking to Your Browser

We probably want this lists of apps to link to the Fly.io dashboard. However, clicked linked in the app follow those links as if the menu itself is a browser! So, we need a way to interact with the system and open a site in the system default browser.

Mac and other systems (presumably Windows has an equivalent) have an open command that can open files, URL’s, and whatever else you throw at it. Running open https://google.com, for example, will open that site in your default browser. We can basically shell out to that command from our app.

To do that, we can call Shell::openExternal($url) (see here). This requires that we run some PHP, but our interactions occur in our “browser”, so we need some Javascript!

I opted to use our good friend Livewire.

composer require livewire/livewire "^3.0@beta"

php artisan make:livewire Apps

Now we have files app/Livewire/Apps.php and resources/views/livewire/apps.blade.php.

We can leave our old route and controller. The new resources/views/apps.blade.php is just:

    <x-layout>
    <livewire:apps />
</x-layout>

  

Our old controller app/Http/Controller/AppController.php no longer calls the Fly API, instead it just returns the above view.

The new Livewire component does all the work:

    namespace App\Livewire;

use App\Fly;
use Livewire\Component;
use Native\Laravel\Facades\Shell;

class Apps extends Component
{
    public $apps;

    public function mount()
    {
        $this->apps = (new Fly)->getApps();
    }

    public function openApp($app)
    {
        Shell::openExternal("https://fly.io/apps/".$app);
    }

    public function render()
    {
        return view('livewire.apps');
    }
}

  

On mount, it makes the API call to Fly.io.

We also have a new openApp() method, which calls NativePHP’s Shell facade. This, in turn, runs the open command on the URL we pass it, which instructs our Mac to open the site in our default browser.

We can then update our view to make each item a link. When we click that link, we’ll call this Livewire method openApp(). Here’s file resources/views/livewire/apps.blade.php:

    <div>
    <ul class="...">
        @foreach($apps as $app)
            <li
                class="..."
                wire:key="{{ $app['name'] }}"
            >
                <a
                    wire:click.prevent="openApp('{{ $app['name'] }}')"
                    class="..."
                >
                    <!-- all the rest of the fancy HTML here -->
                    <!-- see https://github.com/fideloper/nativephp-fly -->
                </a>
            </li>
        @endforeach
    </ul>
</div>

  

So, everything looks the same, but now we can click on our apps and see the app in Fly.io’s dashboard in our (real) browser!

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

UX Improvements

We have a bit of UX that’s still gross:

We don’t make the API calls until the menu is first open, which causes a delay
When we re-open the menu later, it doesn’t update itself

There are a bunch of possible solutions for this, with some trade-off between graceful UX and the number of API calls, but what I opted for is:

Show a loading animation when the menu is first open, and grab the data then
Make periodically updating the list your problem to figure out 😘

Is it awkward to show a loading animation on a desktop app? Sorta!

One other idea is to load the data when the app first spins up (in a service provider) and “cache” it for later. I’d use the database cache to keep everything in the local SQLite database that NativePHP sets up for you. But I’m not doing all the work here - we’re just learning NativePHP, not solving easy-but-tedious programming challenges.

So, we already have Livewire. Let’s make the API call after we open the menu, and add a loading animation while it gets the data.

Our updated component:

    <?php

namespace App\Livewire;

use App\Fly;
use Livewire\Component;
use Native\Laravel\Facades\Shell;

class Apps extends Component
{
    public $apps;

    public function loadApps()
    {
        $this->apps = (new Fly)->getApps();
    }

    public function openApp($app)
    {
        Shell::openExternal("https://fly.io/apps/".$app);
    }

    public function render()
    {
        return view('livewire.apps');
    }
}

  

We no longer make the API call on mount() but instead on a method call loadApps().

We can update the view to call the loadApps() method on “init” (after it’s mounted). This way we can show the loading animation while it’s making the API call:

    # File resources/views/livewire/app.blade.php
# Full thing at https://github.com/fideloper/nativephp-fly
<div wire:init="loadApps">
    @if(! is_array($apps))
        <div class="...">
            <svg class="animate-spin ...">
                <!-- snip -->
            </svg>
        </div>
    @else
        <ul class="...">
            @foreach($apps as $app)
            <!-- do the needful -->
            @endforeach
        </ul>
    @endif
</div>

  

Note the addition of wire:init="loadApps", and we add a conditional to optionally show a loading animation.

We now have a “complete” menu app! We saw how to use NativePHP to quickly make a Menu App, and got a feel for how Laravel translates into a Desktop application.

FilamentPHP: Shooting lasers at the moon

2023-08-15T00:00:00+00:00

Did you know that there are reflectors on the moon? The Apollo missions left them on the surface of the moon for scientists on earth to shoot lasers at. By measuring the time it takes for the laser to go to the moon and back, the scientists can calculate the distance from the earth to the moon with millimeter precision. This led to the discovery that the moon is spiraling away from the Earth. This experiment was called the Lunar Laser Ranging experiment, and we’re going to do a software equivalent of it!

Take a look at the finished dashboard in my Filament app:

Here, you can see the regions where I have an instance of a reflector app running. I’ll be using some fancy networking magic provided by your friends at fly.io to measure how long it takes to send a simple message back and forth.

I wonder if you can calculate where I live from this data? (Please don’t.)

Instead, let’s pop the hood:

The Filament part

Here’s my basic goal: In my database I save data about the Fly Regions, and about the Machines my reflector app is running on. The Machine model is contains a machine_id (the ID of the machine on Fly.io) and a region_id for the Region relationship. We’ll need the machine_id later on for the networking part.

For every Machine, I wanted to show a widget on the Dashboard page use Header Widgets. I ran into issues though: For headerWidgets to work in Filament, you need to configure the classnames of the widgets you’ll use. Making a widget at runtime for each machine and sharing with each widget what machine it belongs seemed difficult and hacky.

So, I made my own custom Filament page with php artisan make:filament-page <page-name-here> with a foreach loop that loops over the machines and shows a card for every machine. Quick tip: just use <x-filament::card></x-filament::card> to use Filament’s excellent styling.

Then, I just needed to add the machines. Since I’m very lazy, I added an Action on the ListMachines index page to fetch all the machines and add them to the database. Here’s how it looks:

    Actions\Action::make('fetchMachines')
    ->action(function() {
        $response = Http::withToken(env('FLY_AUTH_TOKEN'))
            ->get('https://api.machines.dev/v1/apps/fly-filament-satellite/machines');
        $machines = json_decode($response->body());

        //delete machines with different ID
        $machineIDs = array_column($machines, 'id');
        Machine::whereNotIn('machine_id', $machineIDs)->delete();

        // create the new machines
        foreach($machines as $machine)
        {
            $attributes = ['machine_id' => $machine->id, 'region_id' => Region::where('iata_code', $machine->region)->first()->id];
            Machine::firstOrCreate($attributes);
        }
    })

  

This makes a call to the Fly.io machines API to get all the machines for the app ‘fly-filament-satellite’, our reflector app. To authenticate on the machines API, you’ll need an auth token. You can get this by running fly auth token in your terminal. When I use it in Laravel apps, I usually save it in an environment variable locally or as a Fly Secret in apps running on Fly.io.

The closest region

On my dashboard, I show what the closest region is. This is the region where the request was accepted in and will be routed from. This can be any one of Fly.io’s regions, your app doesn’t need to be running there. The request will come into Fly.io’s network on that region and will be routed to the closest region where your app has a machine running.

My closest region is London, UK. In the example above, there’s an app running in only one region: Amsterdam, the Netherlands. My request would arrive on the edge node in London, which will send the request on to the app running in Amsterdam. The response would then return to me over the edge node in London.

Whenever an edge node passes on a request to an app instance, it adds some headers with useful info. Here’s how a request to https://api.machines.dev/v1/apps/[appname here] came back:

On the end of fly-request-id, there’s lhr. This means my request entered the Fly.io network in London. Since all the fly regions are already in the database, we can easily connect a region code to the corresponding region. Cool!

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

The latency

This is the shooting-lasers-at-the-moon part you’ve been waiting for. Don’t worry, it doesn’t involve rocket science.

Let’s take a look at our reflector app first: This app will just respond OK to every request we throw at it. But that’s not all: we need to be able to control what exact region handles our requests. We don’t want the Fly Proxy picking the closest region to us!

Enter dynamic request routing™️: if our reflector app sets a fly-replay header on the response, the Fly Proxy will replay the original request instead of sending the response to the client. If we put a region in the fly-replay header, the original request will be replayed in the specified region. Neat! To let our reflector app know what region to replay the request in, we’ll use a route parameter. So, if we send a request to https://fly-filament-satellite.fly.dev/api/ping/nrt we want the request to be replayed to Tokyo (nrt).

Let’s look at this more closely:

The closest region to me is London so that’s where the request will enter the network, but the closest app instance is in Amsterdam. This is what will happen: The London edge node will pass the request on to the Amsterdam region, since that’s geographically the shortest distance. There, the app will notice we want the request to be replayed in Tokyo because of the nrt route parameter. The app will send a response that contains this header: fly-replay: region=nrt. The Fly proxy will pick this up, and see the response should not go to the client. Instead, it will replay the original request to the nrt region.

One small issue: If we always set a fly-replay header, we’ll create an infinite loop. Luckily for us, when a request has been replayed there’ll be a fly-replay-src header that shows that the request has been replayed and where it originates from. So, if that header is present on the request we just omit the fly-replay header, so the request won’t be replayed then. Cool!

Here’s how the only controller on my reflector app looks:

    // Route::get('/ping/{region}', PingMachine::class)->name('ping-machine');
public function __invoke(string $region, Request $request)
{
    if ($request->hasHeader('fly-replay-src'))
    {
        return response([], 200, []);
    }
    else
    {
        return response(['current-time' => microtime(true)], 200, [
            'fly-replay' => 'region=' . $region,
        ]);
    }
}

  

So, whenever we send a request to /ping/nrt , it’ll be replayed to the machine running in Tokyo which will then respond to the client.

Now, all we need to do is send a request for every region our app runs in, and log the time it takes for the response to get back to us. Here’s the function that’s behind the ‘ping all’ button on the dashboard:

    public function pingAllMachines()
{
    foreach($this->machines as $machine)
    {
        $response = Http::get('https://fly-filament-satellite.fly.dev/api/ping/' . $machine->region->iata_code);
        $this->latencies[$machine->machine_id] = $response->transferStats->getTransferTime();
    }
}

  

Using transferStats->getTransferTime() we can easily get the time between sending the request and receiving the response.

There we go, we recreated the Lunar Laser Ranging experiment with Fly Machines. Unlike the moon though, the Fly Machines won’t spiral away from us. I’m sure you agree that’s a good thing.

Livewire 3 Forms

2023-08-14T00:00:00+00:00

Fly.io can build and run your Laravel apps globally. Deploy your Laravel application on Fly.io, you’ll be up and running in minutes!

Forms are why I use Laravel. Anything that abstracts away the tedium of form security, validation, user-messaging (“hey this is required”), and saving state is my favorite thing. This is, in fact, a huge part of what keeps me using Laravel over any other framework.

Livewire has always made Forms easy to work with. In Livewire 3, it gets even better with a handy abstraction (or perhaps we’ll call this an “extraction”).

Similar to how Laravel has custom Request objects, InertiaJS has a form helper, and Spatie has (had?) that amazing-for-its-time frontend form validation, Livewire 3 now has a form abstraction.

This cleans up your Component code a ton - form input validation / handling in any web app is so routine, so rote, so tediously omnipresent that it’s nice to have it be easy AND tucked out of the way - giving the more interesting code the chance to stick in our brain in our flow state.

Forms

Instead of jamming form “stuff” into our Livewire component, we can create a new Form object:

php artisan livewire:form SnackRequestForm

We can take that form object and add the fields / validation rules:

    namespace App\Livewire\Forms;

use Livewire\Attributes\Rule;
use Livewire\Form;

class SnackRequestForm extends Form
{
    #[Rule('required|in:sweet,salty,hot,cold,disgusting')]
    public $type = '';

    #[Rule('required|in:asap,morning,afternoon')]
    public $time = '';
}

  

Then, in our component, we can use that form:

    namespace App\Livewire;

use Livewire\Component;
use App\Models\SnackRequest;
use App\Livewire\Forms\SnackRequestForm;

class SnackRequest extends Component
{
    public SnackRequestForm $form;

    public function save()
    {
        SnackRequest::create(
            $this->form->all()
        );

        return $this->redirect('/requests');
    }

    public function render()
    {
        return view('livewire.request-snack');
    }
}

  

Finally, your inputs in the HTML form would reference the form itself, e.g. your input list of possible snack types would have wire:model="form.type" instead of wire:model="type". The same goes for validation errors, e.g. you can use @error('form.type') {{ $message }} @enderror if an invalid type was passed.

    <form wire:submit="save">
    <select wire:model="form.type">
        <option value="disgusting">Disgusting</option>
        <!-- and so on -->
    </select>
    <div>
        @error('form.type') <span class="error">{{ $message }}</span> @enderror
    </div>
    <!-- and so on -->
    <button type="submit">Save</button>
</form>

  

Now the more boring form stuff is tucked away.

Validation

Our form right now won’t actually do any validation - we need to use one of 2 options to get it validating.

The first way is to just call $this->validate() in our save() method:

    public function save()
{
    $this->validate();

    SnackRequest::create(
        $this->form->all()
    );

    return $this->redirect('/requests');
}

  

This method will validate only on form submit.

However, we could instead make our model bindings “live”. Adding this makes it so updating field values sends a request across the wire (thus updating the field value “live”), which gives the backend a chance to validate it.

This works well with select lists in a simple form like ours. To use it, we append .live to wire:model in our blade template:

    <form wire:submit="save">
-    <select wire:model="form.type">  
+    <select wire:model.live="form.type">
        <option value="disgusting">Disgusting</option>
        <!-- and so on -->
    </select>
    <div>
        @error('form.type') <span class="error">{{ $message }}</span> @enderror
    </div>
    <!-- and so on -->
    <button type="submit">Save</button>
</form>

  

Note the diff there where we append .live so it becomes wire:model.live="foo".

Further Refactoring

Since the Form object is just a class, you could add methods to it. Perhaps you could have the form save itself?

The component:

    public function save()
{
+    $this->form->submit();
-    SnackRequest::create(
-        $this->form->all()
-    );

    return $this->redirect('/requests');
}

  

And then in the form object:

    public function submit()
{
    SnackRequest::create(
        $this->all()
    );
}

  

That’s (potentially) tidier, depending on what data/logic your form needs when performing needed actions. You’ll need to decide if you like this “pattern” or not.

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

How’s It Work?

As I looked into the new form objects, I realized it wasn’t obvious when validation happened!

You can call $this->validate() anytime within the component. However, validation also happens automagically whenever a mutation happens to one of the public properties (e.g. when you wire:model.live="form.type" and make a change to that input field).

We needed to add the .live modifier to trigger the validation, however, as otherwise Livewire doesn’t see the value of the field changing on the backend and thus wouldn’t validate it.

Digging a bit deeper - the form objects all extend this empty (but nicely namespaced) class which in turn extends this internal base class.

The new form objects collects older-style, component-defined validation rules (returned via method rules() on a component) and finds properties with attribute based validation defined within the form object. These get merged together, so validation happens smoothly and is backwards compatible from Livewire 2.

Chat Widget with Livewire 3's Persist

2023-08-07T00:00:00+00:00

Fly.io can build and run your Laravel apps globally, which is especially good for Livewire! Deploy your Laravel application on Fly.io, you’ll be up and running in minutes!

We’re going to create a new app, add Livewire 3, and show how to persist a chat widget while navigating through the app.

Here’s a GitHub repository with the final code.

composer create-project laravel/laravel livewire3-persist
cd livewire3-persist

composer require livewire/livewire:^3.0@beta calebporzio/sushi

✨ We no longer need to update our layout file to include livewire JS and styles - that’ll happen automatically (unless configured not to). Note, however, that it only happens automatically on pages with certain livewire things happening on it, such as a component being used, or the new @persist option present.

We’ll create a quick layout at resources/views/components/layout.blade.php:

    <!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>My Video Collection</title>
    <script src="https://cdn.tailwindcss.com?plugins=typography"></script>
</head>
<body class="min-h-screen bg-gray-50 font-sans text-black antialiased">
{{ $slot }}
</body>
</html>

  

Then we can create a “home” page at resources/views/home.blade.php, a video list component at resources/views/components/videos.blade.php, and then a single-video page at resources/views/video.blade.php.

None of that is Livewire - plain old Laravel so far!

Let’s create a Livewire component - it will be a chat widget. Our goal is to persist this chat widget throughout the site without losing any state as we navigate the site.

php artisan livewire:make Chat

This creates file app/Livewire/Chat.php and resources/views/livewire/chat.blade.php.

✨ Note that components are no longer in the Http namespace!

There’s nothing fancy about this chat widget. In fact, all we’re doing for this demonstration is letting a user input their side of the conversation. Drawing the rest of the owl is your job.

What we care about is persisting the chat state.

Here’s the component:

    <?php

namespace App\Livewire;

use Livewire\Component;

class Chat extends Component
{
    /**
     * @var string[]
     */
    public array $messages = [];

    public string $message = '';

    public function addMessage()
    {
        $this->messages[] = $this->message;
        $this->reset('message');
    }

    public function render()
    {
        return view('livewire.chat');
    }
}

  

We have an array of $messages. Anytime we addMessage(), we just append to this list of messages.

On the HTML side, we can see how addMessage() is called, providing the value $message via a form submit.

    <div
    class="absolute bottom-0 right-12 h-60 w-60"
>
    <div class="w-full h-full bg-white border rounded overflow-auto flex flex-col">
        <div
            x-ref="chatBox"
            class="flex-1 p-4 text-sm flex flex-col gap-y-1">
            <div class="text-gray-400 italic">Chat history</div>
            @foreach($messages as $message)
                <div><span class="text-blue-400">You:</span> {{ $message }}</div>
            @endforeach
        </div>
        <div>
            <form
                wire:submit="addMessage"
            >
                <input 
                    wire:model="message" 
                    x-ref="messageInput" type="text" name="message" id="message" 
                    class="block w-full rounded-md border-0 py-1.5 text-gray-900 shadow-sm ring-1 ring-inset ring-gray-300 placeholder:text-gray-400 focus:ring-2 focus:ring-inset focus:ring-indigo-600 sm:text-sm sm:leading-6"
                />
            </form>
        </div>
    </div>
</div>

  

This chat widget is absolutely positioned, pinned to the bottom of the screen. Since we want it on every page, we’ll add it to our layout file:

    <!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>My Video Collection</title>
    <script src="https://cdn.tailwindcss.com?plugins=typography,forms"></script>
</head>
<body class="min-h-screen bg-gray-50 font-sans text-black antialiased">
    <header>
        <nav class="mx-auto max-w-2xl flex items-center p-6">
          <a href="/">Home</a>
        </nav>
    </header>
    {{ $slot }}

    <livewire:chat />
</body>
</html>

  

We added some navigation with a home button, and included our new <livewire:chat /> component.

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

And yet, it persisted

Our chat widget doesn’t persist just yet! If we add some content, but navigate away, the chat component is reloaded (as you’d expect).

To make the state persist, we need 2 new tricks in Livewire 3:

wire:navigate
@persist

✨ Any anchor tags <a> that we create can include the wire:navigate attribute. This puts the app into “SPA mode”, which works like Turbolinks..er…Turbo (if you’re familiar with it). It uses Javascript to load the requested page, and replaces the <body> content with the result. The browser actually thinks it’s the same page, but JS changed the <body> content, the URL, and the <title> tag. This helps us create an SPA-like experience.

✨ On top of this, we can use the new @persist blade directive! If Livewire sees a section wrapped in @persist('some-name') (and sees the named block in the result of the next request), it will keep that DOM element within the <body> tag untouched.

Here’s the updated layout file to make that work:

    <!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>My Video Collection</title>
    <script src="https://cdn.tailwindcss.com?plugins=typography,forms"></script>
</head>
<body class="min-h-screen bg-gray-50 font-sans text-black antialiased">
    <header>
        <nav class="mx-auto max-w-2xl flex items-center p-6">
          <a href="/" wire:navigate>Home</a>
        </nav>
    </header>
    {{ $slot }}

    @persist('chat')
    <livewire:chat />
    @endpersist
</body>
</html>

  

Note the addition of wire:navigate and @persist('chat'). It’s not shown in the snippet above, but I added wire:navigate to all anchor tags in this tiny app.

Now when we navigate around the app, the chat widget and its state persists!

We have an SPA without all the cruft of an SPA!

Checking out toRawSql

2023-07-25T00:00:00+00:00

Fly.io can build and run your Laravel apps globally, including your scheduled tasks. Deploy your Laravel application on Fly.io, you’ll be up and running in minutes!

In the recent past, we were able to dump out the SQL our query builder was generating like so:

    $filter = 'wew, dogs';

// Using the `toSql()` helper
DB::table('foo')
  ->select(['id', 'col1', 'col2'])
  ->join('bar', 'foo.bar_id', 'bar.id')
  ->where('foo.some_colum', $filter)
  ->toSql();

// SELECT id, col1, cole2
//     FROM foo
//     INNER JOIN nar on foo.bar_id = bar.id
//     WHERE foo.some_column = ?

  

This was useful for debugging complicated queries, but note how we didn’t get the value in our WHERE statement! All we got was a pesky ? - a placeholder for whatever value we’re passing into the query. The actual value is hidden from us.

“That’s not what I need”, you may have asked yourself. Assuming we aren’t debugging SQL syntax, our query bindings are what we likely care about the most.

Getting the Full Query

New to Laravel 10.15 is the ability to get the full sql query! That’s much more useful for debugging.

    $filter = 'wew, dogs';

// Using the `toRawSql()` helper
DB::table('foo')
  ->select(['id', 'col1', 'col2'])
  ->join('bar', 'foo.bar_id', 'bar.id')
  ->where('foo.some_colum', $filter)
  ->toRawSql();

// SELECT "id", "col1", "col2"
//     FROM "foo" 
//     INNER JOIN "bar" ON "foo"."bar_id" = "bar"."id"
//     WHERE "foo"."some_colum" = 'wew, dogs'"

  

Much better!

The trick to this is that PDO (the core library used to connect to databases) doesn’t just give this to us - we can only get the SQL with the binding placeholders (hence the old toSql() limitation).

So, we need to build the query with our values within the query ourselves! How’s that done?

How It’s Done

This is tricky business, as we’re dealing with user input - any crazy thing a developer (or their users) might throw into a sql query needs to be properly escaped.

The new toRawSql() helper stuffs the important logic into a method named substituteBindingsIntoRawSql(). Here’s the PR, for reference.

If we dig into that method code a bit, we’ll see what’s going on!

The first thing the function does is escape all of the values. This lets Laravel print out the query as a string without worrying about mis-aligned quotes or similar issues.

    $bindings = array_map(fn ($value) => $this->escape($value), $bindings);

  

The call to $this->escape() goes down to a database connection object, and deeper into the underlying PDO object. PDO does the work of actually escaping the query values in a safe way.

You can get a “Connection Refused” error using the toRawSql() method if your database connection isn’t configured or isn’t working. That’s because the underlying code uses the “connection” object (PDO under the hood) to escape characters within the output.

Following the escaping-of-values, the method goes through the query character by character!

    for ($i = 0; $i < strlen($sql); $i++) {
    $char = $sql[$i];
    $nextChar = $sql[$i + 1] ?? null;

    // and so on
}

  

The major supported databases use different conventions for escape characters. The code here attempts to find escaped characters and ignore them, lest it tries to substitute something that looks like a query binding character but isn’t. This is the most fraught bit of code in this new feature.

    $query = '';


for ($i = 0; $i < strlen($sql); $i++) {
    $char = $sql[$i];
    $nextChar = $sql[$i + 1] ?? null;

    // Single quotes can be escaped as '' according to the SQL standard while
    // MySQL uses \'. Postgres has operators like ?| that must get encoded
    // in PHP like ??|. We should skip over the escaped characters here.
    if (in_array($char.$nextChar, ["\'", "''", '??'])) {
        // We are building the query string back up - We ignore escaped characters
        // and append them to our rebuilt query string. Since we append
        // two characters, we `$i += 1` so the loop skips $nextChar in our `for` loop
        $query .= $char.$nextChar;
        $i += 1;
    } ...
}

  

The for loop is rebuilding the query string, but with values substituted in for their ? placeholders. The first check here is looking for certain escape characters. It needs to know the current character AND the next one to know if it’s an escaped character and therefore should not do any substitutions.

The next part of our conditional is this:

    } elseif ($char === "'") { // Starting / leaving string literal...
    $query .= $char;
    $isStringLiteral = ! $isStringLiteral;
}

  

If we’re opening an unescaped quote, it means we’re at the start (or end) of a string literal. We set a flag for this case, which is important in our next check.

    elseif ($char === '?' && ! $isStringLiteral) { // Substitutable binding...
    $query .= array_shift($bindings) ?? '?';
}

  

Here’s the magic. If we’re NOT inside of a string literal, AND we’re not finding an escaped character, AND we find a ? character, then we can assume it’s a query binding to be substituted with the actual value. We take our array of values and shift it - we remove the first item in that array and append its value to our query string. (The array of values are in the same order of query binding characters - ? - in the query).

Finally, if we just have a regular character that doesn’t have special meaning, we just append it to our query string:

    else { // Normal character...
    $query .= $char;
}

  

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Here’s the whole method, as it stands as I write this:

    public function substituteBindingsIntoRawSql($sql, $bindings)
{
    $bindings = array_map(fn ($value) => $this->escape($value), $bindings);

    $query = '';

    $isStringLiteral = false;

    for ($i = 0; $i < strlen($sql); $i++) {
        $char = $sql[$i];
        $nextChar = $sql[$i + 1] ?? null;

        // Single quotes can be escaped as '' according to the SQL standard while
        // MySQL uses \'. Postgres has operators like ?| that must get encoded
        // in PHP like ??|. We should skip over the escaped characters here.
        if (in_array($char.$nextChar, ["\'", "''", '??'])) {
            $query .= $char.$nextChar;
            $i += 1;
        } elseif ($char === "'") { // Starting / leaving string literal...
            $query .= $char;
            $isStringLiteral = ! $isStringLiteral;
        } elseif ($char === '?' && ! $isStringLiteral) { // Substitutable binding...
            $query .= array_shift($bindings) ?? '?';
        } else { // Normal character...
            $query .= $char;
        }
    }

    return $query;
}

  

That’s basically all there is to the story. We want the entire query to be available with our values! Here are some (light) caveats we have to get this feature:

We need to make a database connection (thanks to using the PDO library for safe escaping)
The above code MAY contain the occasional bug depending on what values are used
Very long (e.g. binary) values in a query would likely return a complete mess of a query string

Those trade-offs seem fine to me!

Checking Out Sub-Minute Scheduling

2023-07-24T00:00:00+00:00

Fly.io can build and run your Laravel apps globally, including your scheduled tasks. Deploy your Laravel application on Fly.io, you’ll be up and running in minutes!

One of Laravel’s new, fun enhancements is the ability to run scheduled commands (or jobs) more than once per minute!

Notoriously, CRON can only run tasks every 1 minute or slower. Laravel’s scheduler, typically driven by CRON, therefore only ran once per minute.

Well, that hasn’t changed! It’s still powered by CRON, but now it can run tasks more than once per minute. That fancy thing powering that?

A while loop.

How?

When the scheduler runs once per minute, it finds and runs tasks that are due. With sub-minute tasks, it now has an additional while() loop that runs for that entire minute.

    # First, run all due tasks
foreach ($events as $event) {
  // run events as normal
}

# Directly after, find tasks that should
# repeat within the current minute
while (Date::now()->lte($this->startedAt->endOfMinute())) {
  foreach ($repeatableEvents as $event) {
    // Run repeatable events
  }
}

  

Each while-loop iteration, it checks to see if there are “events” (also called “tasks” - these are scheduled commands or jobs) that are due to run. So, if we have a command that should run every 30 seconds, the scheduler will run that twice within the given minute.

If there are no more things it might possibly run, it is smart enough to exit out of that new loop.

The PR for this feature is here.

The top-level code is in the schedule:run command’s handle() method. It finds events that are “due now” and then runs them. That’s the normal thing the scheduler always did.

However, after that, it now also checks for any events that should be repeated. That’s where our new while() loop comes in.

Here’s a selection of code from the scheduler’s handle() method, with some added comments:

    // Illuminate/Console/Scheduling/ScheduleRunCommand

public function handle(...)
{
    $events = $this->schedule->dueEvents($this->laravel);

    // Run regular events due now (at the top of the minute)
    foreach ($events as $event) {
        $this->runEvent($event);
    }

    // Repeatable events are ones that can happen 
    // more than once per minute, and thus repeat within
    // this run of `artisan schedule:run`
    if ($events->contains->isRepeatable()) {
        // Note the filter
        $this->repeatEvents($events->filter->isRepeatable());
    }
}

  

You’ll note that Laravel runs “normal” tasks BEFORE doing any repeatable ones. Everything is, by default, run in sequence - so any commands (or jobs, if not sent to the queue) that take a long time to run will delay the running of sub-minute tasks.

You can push some tasks to the background if they take too long.

In any case, if repeatable events are found, then we call repeatEvents(), which is the home of our new while() loop.

    // Illuminate/Console/Scheduling/ScheduleRunCommand

protected function repeatEvents($events)
{
    // Only run within the current given minute
    while (Date::now()->lte($this->startedAt->endOfMinute())) 
    {
        // These events have been pre-filtered to only include
        // sub-minute tasks
        foreach ($events as $event) {
            // Only run events ready to be repeated
            if (! $event->shouldRepeatNow()) {
                continue;
            }

            $this->runEvent($event)
        }
    }
}

  

Keep in mind I’m simplifying the code there, which means there are important checks / features I’m not showing (running or not running in maintenance mode, signals to interrupt the loop, etc).

The gist is that this just runs within the current minute, and any “repeatable” task gets run when they’re due. Events that run every 30 seconds get run twice - once in the first foreach() loop of the handle() method, and again in this while loop that is looking for repeatable events.

Timing (checking if a job needs to be run) logic hasn’t really changed here - Laravel was always comparing an “event” that’s scheduled versus the current time. If an event’s timing is less-than-or-equal-to-now, then it’s ready to be run.

Sub-minute events are the same, we just needed the schedule:run command to run for the entire minute and continuously check to see if any repeatable events are due.

Interrupt

There’s also a new command artisan schedule:interrupt, which will gracefully stop the while() loop if the interrupt is flagged.

This is useful for deployments, so we don’t stop/shut off a running task when it’s mid-process. If you use sub-minute tasks, then your deploy process should include a call to artisan schedule:interrupt.

New Options

The new options available when scheduling tasks are:

    $schedule->job(new ExampleJob)->everySecond();
$schedule->job(new ExampleJob)->everyTwoSeconds();
$schedule->job(new ExampleJob)->everyFiveSeconds();
$schedule->job(new ExampleJob)->everyTenSeconds();
$schedule->job(new ExampleJob)->everyFifteenSeconds();
$schedule->job(new ExampleJob)->everyTwentySeconds();
$schedule->job(new ExampleJob)->everyThirtySeconds();

  

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Time Cop

You can’t currently set an arbitrary number of seconds, as certain intervals are weird to process when dividing how many times per minute the task should run.

Imagine running a task every 7 seconds! You’d end up with a task run ~8.5 times per minute (which would essentially be 8 times). Laravel skips over this possibility by disallowing times that might result in this situation like so:

    protected function repeatEvery($seconds)
{
    if (60 % $seconds !== 0) {
        throw new InvalidArgumentException("The seconds [$seconds] are not evenly divisible by 60.");
    }

    $this->repeatSeconds = $seconds;

    return $this->everyMinute();
}

  

You can see the exception thrown if there’s a remainder when dividing into 60.

Another fun thing: The repeatEvery() method sets an attribute “repeatSeconds” with the seconds value, and then sets the task to run every minute!

This gets the task to run at the top of the minute. Afterwards, the while() loop takes the repeated task and runs it again, if the repeatSeconds attribute is set (and when the task is due to run).

This is a neat feature! Like anything that seems super powerful, the execution has a lot of details, but usually isn’t as complex as you might think on the surface.

FilamentPHP: Adding some style

2023-07-18T00:00:00+00:00

Fly.io can build and run your Laravel app globally, on our own hardware. Deploy your Laravel application on Fly.io, you’ll be up and running in minutes!

Let’s be real here: FilamentPHP has great styling out of the box, using tailwindcss and a coherent set of components. While beauty may be subjective, it’s a clear and inoffensive style anyone can appreciate.

But let me ask you this: How do you react when you see a basic web app that uses bootstrap’s default styling? For me, I’m always turned off by it and while it doesn’t affect the app’s functionality, it does give me a slightly worse impression.

So, let’s update the styling of our basic Filament app! I’ll start with some easy stuff and I’ll stray further away from the default the further we move along. Read until the end to see me override the default components and bend them to my will, superhuman-style!

Here’s a before and after of the app:

Notice the subtle background pattern?

A splash of color

First easy change: Change the color theming!

I’ll change the primary color from Filament-orange to a Fly.io-violet. This uses themes, so we’ll need to install some tailwind stuff with npm first: npm install tailwindcss @tailwindcss/forms @tailwindcss/typography autoprefixer tippy.js --save-dev . Once that’s done, create a tailwind.config.js file by running npx tailwindcss init. In there, we can set it up like this:

    /** @type {import('tailwindcss').Config} */

const colors = require('tailwindcss/colors')

module.exports = {
    content: [
        './resources/**/*.blade.php',
        './vendor/filament/**/*.blade.php'
    ],
    theme: {
        extend: {
            colors: {
                danger: colors.rose,
                primary: colors.violet,
                success: colors.emerald,
                warning: colors.amber,
            },
        },
    },
    plugins: [
        require('@tailwindcss/forms'),
        require('@tailwindcss/typography'),
    ],
}

  

Then, add this to the boot() method of a service provider. I’d suggest creating a FilamentServiceProvider to group all the Filament stuff together.

    Filament::serving(function () {
    Filament::registerViteTheme('resources/css/filament.css');
});

  

After that we’ll need to add the filament.css file to vite.config.js:

    // vite.config.js
laravel({
    input: [
        'resources/css/app.css',
        'resources/js/app.js',
+       'resources/css/filament.css'
        ],
    refresh: true,
}),

  

Create postcss.config.js in your project root and add the following:

    export default {
    plugins: {
        tailwindcss: {},
        autoprefixer: {},
    },
}

  

And finally, create a resources/css/filament.css file and import Filament’s vendor css:

@import '../../vendor/filament/filament/resources/css/app.css';

If we reload our app, we’ll see that all the colors have been modified! Now, let’s change the fonts.

Font-astic choice!

The font you use always plays a huge part in the look and feel of the application. I’ll use Quicksand which is a nice rounded sans-serif font.

First up we’ll need to publish the Filament configuration: php artisan vendor:publish --tag=filament-config. After that we can change the google fonts link in config/filament.php:

    'google_fonts' => 'https://fonts.googleapis.com/css2?family=Quicksand:wght@400;500;700&display=swap',

  

Then you can add the fonts to the theme in tailwind.config.js:

theme: {
    extend: {
        colors: {
            danger: colors.rose,
            primary: colors.violet,
            success: colors.emerald,
            warning: colors.amber,
        },
+       fontFamily: {
+           sans: ['"Quicksand"', 'sans-serif'],
+       }
    },
},

Reload the page and boom: New font, new me!

Fly.io ❤️ Filament

Provision servers close to your users and serve your Filament app at light speed. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Dark Mode

We can enable dark mode by setting dark_mode to true in config/filament.php . This is the file we published when changing the fonts. When dark mode is enabled, a setting on the user menu will appear to toggle between light and dark mode. Right now, it probably doesn’t do much. Make sure npm run dev is running and then it should work.

Right now, we’ve done some pretty basic stuff that’s all very point-and-click. Let’s get out of the kiddie pool and add a background pattern to the main section of each page, like this:

Background patterns

For the background pattern, all we need to do is override the Base view component and add a pattern to it. Easy peasy!

Ok, maybe let’s rewind a little. Let’s pretend to create a custom Filament page to show you how the blade component discovery works. We’d use php artisan make:filament-page demo which would create a demo.blade.php page along with a Demo php class. The blade file would look like this:

<x-filament::page>

</x-filament::page>

So how does the blade engine know where to find x-filament::blade? This is an example of component namespaces. The laravel docs explain it briefly here: https://laravel.com/docs/10.x/blade#manually-registering-package-components

Filament doesn’t exactly use the componentNamespace method in a service provider, but the concept is exactly the same: By providing a namespace and a prefix, the Blade knows what directory to look in when a certain prefix is used. This is what it would look like for the Filament package:

Blade::componentNamespace('Filament//views//components', 'filament')

So, whenever <x-filament::component> is used, the Blade engine will look in vendor/filament/filament/resources/views/components/ for a component.blade.php file. But before it does that, the Blade engine will check in your app’s resources/views folder if it can find the blade component there. This means that if we create our own files in a certain directory, the Blade engine will use our files instead of those in the vendor folder!

So to override Filament’s Base component, create a base.blade.php file in the resources/views/vendor/filament/components/layouts folder. For clarity, this is what it would be used like: <x-filament::components.layouts.base>.

Then, just copy over everything from the base component in Filament’s vendor folder. Then, find the opening tag of the html body, it should be on line 108. We’ll add a custom tailwind class in there called bg-topography-pattern :

    <body
    @class([
        'filament-body min-h-screen overflow-y-auto bg-gray-100 fill-cyan-500 bg-topography-pattern text-gray-900',
        'dark:bg-gray-900 dark:text-gray-100' => config('filament.dark_mode'),
    ])
>

  

The topography pattern is a pattern I’ve downloaded from https://heropatterns.com using the ‘download unstyled svg’ button. Make sure to put the svg file in your public/images folder.

The default svg will have black lines which is not really vibing with the ‘subtle background pattern’ feeling I’m looking for. So, on the <path> tag I changed the fill from #000 to #f3e8ff which corresponds to purple-200 in tailwind country.

Lastly, we can add a custom tailwind class in our theme to set the svg as the background image. Just extend the theme, like this:

    theme: {
    extend: {
        colors: {
            danger: colors.rose,
            primary: colors.violet,
            success: colors.emerald,
            warning: colors.amber,
        },
        fontFamily: {
            sans: ['"Quicksand"', 'sans-serif'],
        },
+       backgroundImage: {
+           'topography-pattern': "url(/public/images/topography.svg)"
+       }
    },
},

  

This makes a new tailwind class bg-topography-pattern available, which is the one we used in the base component. Right now, your app should have a nice pattern on the background of every page!

On dark mode this pattern will still show up with a very light purple fill. Not a great look. To fix it you can create a dark version of the svg by changing the fill color, adding a dark custom tailwind class and then using that class with the dark: filter on the base component. Fixed!

Extending Laravel with Managers

2023-07-13T00:00:00+00:00

Fly.io can build and run your Laravel app globally. Deploy your Laravel application on Fly.io, you’ll be up and running in minutes!

Most Laravel libraries that have drivers use a Manager class.

A “driver” is an implementation of some library, but (generally) using a different underlying technology.

A few example Laravel libraries with drivers (multiple implementations) are: Queue, Database, Cache, and Session.

To make that a bit more concrete: the drivers you can choose between for the Queue library (out of the box) are Sync, Database, Beanstalkd, SQS, Redis, and Null.

From our perspective (as developers), the queue system is interacted with in the same way. Each driver, however, handles its specific implementation details as dictated by the underlying technology used.

sync runs the jobs immediately (no background work done at all)
database uses database tables and has to care about database-specific locking mechanisms
sqs uses AWS’s code (their HTTP API)
redis uses LUA scripts to run commands against a Redis instance

Managers Manage Implementations

For any library, the aforementioned Manager classes manage the various drivers available. The Manager acts as a place to register and create the implementations of a library.

Let’s look at the DB facade. This facade resolves to whatever is registered as, simply, db. If we check DatabaseServiceProvider, we see that db resolves to an instance of DatabaseManager:

    $this->app->singleton('db', function ($app) {
    return new DatabaseManager($app, $app['db.factory']);
});

  

So, when we use DB::foo(), we’re interacting with the DatabaseManager class.

The DB manager is able to give us registered implementations. For example, two ways to get a DB connection are DB::connection() (to get the default connection) and DB::connection('sqlite') (to get a connection named sqlite).

The connection classes returned have the same set of methods available on them - they implement a shared interface. This means you can call $connection->table('users')->get() on all database connection types.

The DatabaseManager class, like most Manager classes, also has an extend() method that we can use to add our own database drivers.

This is common to all Manager classes within Laravel. They are responsible for letting us extend a library and also generating implementations (acting as a Factory).

Bring Your Own Driver

You can extend a lot of Laravel with your own drivers - and managers are how!

Let’s see how to create an alternative Queue driver. In our case, we’ll extend and tweak the SQS queue driver.

It basically boils down to this:

    # Somewhere within a service provider...
use Illuminate\Queue\QueueManager;

QueueManager::extend('sqs-poll', function() {
    return new SqsLongPollerQueue;
});

  

This adds a new SQS queue driver named sqs-poll. Why do this? Because I want to add long-polling.

For SQS, long polling keeps the SQS connection open for X seconds, waiting for new jobs to come in. The end result is that we make less API calls to SQS. Since we’re billed on SQS calls, this reduces cost. SQS is pretty cheap, so this is only really useful at scale.

You should also keep your queue timeout option at the set number of seconds (or higher) to avoid queue timeout errors polluting your error logs.

Read up on long-polling vs short-polling! Long polling is nice as workers often retrieve jobs sooner - the default “short polling” only asks a subset of SQS servers to see if there is a job! This can delay jobs being processed, even at smaller scales.

Most manager classes have an extend() method that works just like above - you can add a “driver” and define the implementation of it. Then your application can be configured to use it!

To use the above queue driver, I’d update config/queue.php and add a new item to the connections array with array key sqs-poll. In our case, we can copy the sqs driver details for our new sqs-poll connection.

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Our Implementation

For the sake of completeness, let’s see what the SqsLongPollerQueue implementation looks like.

To accomplish what we want, we can extend the SqsQueue class that comes with Laravel, and tweak the pop() method (which is used to get a message from the queue).

    <?php

namespace App\Queue;

use Illuminate\Queue\SqsQueue;
use Illuminate\Queue\Jobs\SqsJob;

class SqsLongPollerQueue extends SqsQueue
{
    /**
     * Pop the next job off of the queue.
     *
     * @param  string|null  $queue
     * @return \Illuminate\Contracts\Queue\Job|null
     */
    public function pop($queue = null)
    {
        $response = $this->sqs->receiveMessage([
            'QueueUrl' => $queue = $this->getQueue($queue),
            'AttributeNames' => ['ApproximateReceiveCount'],
            'WaitTimeSeconds' => 20, // We added this line
        ]);

        if (! is_null($response['Messages']) && count($response['Messages']) > 0) {
            return new SqsJob(
                $this->container, $this->sqs, $response['Messages'][0],
                $this->connectionName, $queue
            );
        }
    }
}

  

It turns out that for this specific use case, you can set the Queue attribute ReceiveMessageWaitTimeSeconds (within AWS), and it will do the same thing - making the code changes above…not needed. Sorry! (Read more on that here).

A Real Example

This method of extension is how the SQS FIFO Laravel package works - it adds the driver (“connection”) sqs-fifo. FIFO stands for “first in, first out” - in other words, unlike with regular SQS, FIFO queues guarantee that jobs are released in the order they are created.

This is useful for workloads that need to be run in a specific order. I use this on Chipper CI to ensure builds are run in the order they are received.

Now, if jobs are run in order, how do you process multiple jobs at the same time? To enable running jobs in parallel, there is a concept of “groups”. A group is just a string used to identify a group of queue jobs. Jobs with group ID “xyz” are all processed in order, while jobs with group ID “abc” will be processed in their own order. Groups “xyz” and “abc” can be run in parallel.

So, back to the FIFO package - the SqsFifoQueue class is the class added via the QueueManager. This class extends SqsQueue (just like our example above) and tweaks what’s needed for FIFO queues to work - allowing us to add job groups and de-duplication checks.

Debugging Tests: A look into assertCommandCalled

2023-07-05T00:00:00+00:00

In just a few steps, deploy your Laravel application on Fly.io, you’ll be up and running in minutes!

Have you ever written a test case, expecting it will pass, only to find out it doesn’t? Today, we’ll create a test for a Laravel Zero console command, and use its assertCommandCalled helper to assert one command calls another.

We’ll then see how easy it is to fall into the pits of despair trying to make an assertion pass, and how thankfully, we can follow a set of steps to get out of it.

The Scenario

Let’s say, we have a Laravel Zero application and it has a scan command. This command checks whether a “config_file” file exists in the base directory, and triggers another command to parse the file if so:

    /* app/Commands/ScanCommand.php */

protected $signature = 'scan';

public function handle(): void
{
  if( file_exists('config_file') ){
    $this->call( ParseConfigCommand::class );
  }
}


  

The command triggered by the scan command has a signature of parse-config, and has the class name ParseConfigCommand. This command will simply provide a command line output indicating it has started, and parse the “config_file” found in the base directory:

    /* app/Commands/ParseConfigCommand.php */
protected $signature = 'parse-config';

public function handle(): void
{
  $this->line( 'Starting parse of config_file' );
  $this->parse();
}

  

Creating the Test

The feature we’ll test out today is whether running the scan command ( given the right conditions ) triggers the parse-config command.

Laravel Zero provides us the helper, assertCommandCalled() in order to assert that a specific command was called. This is perfect for the feature we’re expecting to test. To start, first create a feature test in test/Feature/ScanCommandTest.php:

    /* test/Feature/ScanCommandTest.php */ 
test( 'Scanner calls parse-config command when config_file is found in base directory.', function(){

    // 1. Arrange: prepare config_file in base directory
    file_put_contents( 'config_file', 'data' );

    // 2. Action: run intended command
    $this->artisan( 'scan' );

    // 3. Assert: parse-config was also called
    $this->assertCommandCalled( 'parse-config' );

    // 4. CleanUp
    unlink( 'config_file' );

});

  

There’s a bunch of things happening above, so let’s break it down:

Laravel Zero ships with Pest. And if we check the first line, we’ll see two parameters passed to Pest’s helper function test(). test receives a String description of what is being tested, and a closure containing the actual test expectations.

On to the actual test, notice 4 comments separating our test into 4 important sections:

The first section is Arrangement—this is where necessary preparations are made so that the action we run afterwards exhibit our intended behavior. In our case, we create a “config_file” file in our base directory so that when we run the scan command below, it triggers the parse-config command.
The second section is running the Action we want to test. This is calling the necessary methods, or in our case, triggering the artisan() method to mock-run a console command with the scan signature.
The third section is Asserting our expectation. In our case, we assert that running the scan command (given our preparation) should call the parse-config command.
The final section is clean up. This is the part where we remove any unnecessary, persisted files created thanks to our test, like the “config_file” created from section 1.

Now that we have the different parts of our test covered, let’s move on to running the test.

Running the Test

Thanks to the Arrangement section of our test, the proper file should’ve been created in the base directory before the scan command runs. Because of this, we should expect the scan command to pass its first “if condition”, and therefore call the parse-config command.

So, let’s run the test above with ./vendor/bin/pest, and check the result:

—it failed!

Debugging a Failed Assertion

Why did our assertion fail? If we look at the error message, it says: "Failed asserting that 'parse-config' was called...". Strange. The scan command clearly calls the parse-config command given the proper condition. How do we get to the bottom of this?

First, let’s test the action’s behavior manually. Prepare a “config_file” in the base directory, and run the scan command. Afterwards, check the command line output:

    Starting parse of config_file

  

parse-config‘s line output was printed! This means it is indeed being called by the scan command.

Now, let’s see if it also gets called in a test environment. To do so, we’ll rely on a log entry to record and indicate it’s been called:

    /*  app/Commands/ParseConfigCommand.php  */
+   Log::info( 'Running parse-config command!' );
    $this->parse();

  

Now if we run our test again with vendor/bin/pest and check our log file:

    [2023-07-03 12:13:27] development.INFO: Running parse-config command! 

  

We’ll have our confirmation that parse-config is getting called, even in test!

Neat! We’ve now verified that our action works both manually and in a test environment. This let’s us know that we don’t need to change anything in the code we’re testing.

Even with this confirmation, our assertion still fails however. Now that we’re done with inspecting what’s being tested, let’s move over and inspect how it is being tested.

Making assertCommandCalled() pass

This is how we assert that our parse-config command is called:

    /* test/Feature/ScanCommandTest.php */
$this->assertCommandCalled( 'parse-config' );

And this is a longer version of our error message from the failed assertion above:

    Failed asserting that 'parse-config' was called with the given arguments: ... at `vendor/laravel-zero/framework/src/Testing/TestCase.php:57`

  

If we open the file indicated in the error, at around line 57, we’ll find Laravel Zero’s beautiful declaration of the assertCommandCalled() method:

    /* vendor/laravel-zero/framework/src/Testing/TestCase.php */

protected function assertCommandCalled(string $command, array $arguments = []): void
{
  $argumentsAsString = (new ArgumentFormatter)->format($arguments);
  $recorder = app(CommandRecorderRepository::class);

  static::assertTrue(
    $recorder->exists($command, $arguments),
    'Failed asserting that \''.$command.'\' was called with the given arguments: '. $argumentsAsString
  );
}

  

From our test’s perspective, we give assertCommandCalled() a $command argument of “parse-config”. The method would then feed this value to its $recorder object’s exists() method to see whether the command exists or not.

The $recorder object is an instance of the CommandRecorderRepository class. And if we check what this class is all about, we’ll see that it holds a record of all the commands called in its private property $storage!

This is neat! We finally have a good lead on what’s happening: we can check the commands triggered through the $recorder attribute, and once and for all get to the bottom of our debug journey.

Go ahead, add a quick dd( $recorder ); before the assertion line, and we’ll see:

    ^ LaravelZero\Framework\Providers\CommandRecorder\CommandRecorderRepository^ {#908
  -storage: Illuminate\Support\Collection^ {#971
    #items: array:2 [
      0 => array:3 [
        "command" => "scan"
        "arguments" => []
        "mode" => "default"
      ]
      1 => array:3 [
        "command" => "App\Commands\ParseConfigCommand"
        "arguments" => []
        "mode" => "default"
      ]
    ]
    #escapeWhenCastingToString: false
  }
}

  

See? Indeed the parse-config command is getting called. But, the recorded value is the class name, instead of its signature!

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

If we go back to the declaration of our scan command’s handle method, we do pass a class instance, instead of its signature. Let’s go back to our test case, and add our fix:

    /* test/Feature/ScanCommandTest.php */

$this->assertCommandCalled( 'App\Commands\ParseConfigCommand' );

This should now give us a well deserve pass:

    PASS  Tests\Feature\ScanCommandTest
✓ Scanner calls parse-config command when config_file is found in base directory.               0.04s  

Tests:    1 passed (1 assertions)
Duration: 0.07s

Debugging Tests

A lot has happened above! Here’s a summary of the steps we took in debugging our failed assertion:

First, we made arrangements and run our actions manually. Doing so let’s us see first-hand whether the behavior we intend to see actually happens.
Since the behavior occurs manually, we verify it on our test environment. We re-run the test we created, but rely on a log entry that would indicate to us whether our intended behavior actually occurred behind the scenes.
After double checking our behavior works manually and in the test environment, next up we focus on trying to see how exactly the assertion is being done by checking the method that’s involved with the assertion.
With the assertion method available for inspection, we finally data dumped what values the assertion method was actually receiving.
Finally, thanks to seeing the values, we were able to verify what argument to pass to the assertCommandCalled() method for it to pass our intended assertion.

There’s no sure fire way to debug tests. But, it’s helpful to have a checklist to verify if it’s the code being tested ( steps 1-2 ) or the test itself ( steps 3-5 ) that needs to be adjusted, in order for our test to finally pass.

Handling Signals in Laravel

2023-07-03T00:00:00+00:00

Fly.io can build and run your Laravel app globally. Deploy your Laravel application on Fly.io, you’ll be up and running in minutes!

Laravel queues are stopped gracefully. What does this mean?

During a deployment, you likely restart your queue workers using something like artisan queue:restart or supervisorctl restart <worker-name>.

Laravel has graciously noticed that we don’t like it when an in-process job is suddenly killed. What Laravel does is check to see if a queue worker should stop. If it should, the worker waits until the currently running job is finished, and then exits.

It does this using signals. (We all know that if a page is using 1990’s default HTML formatting, it’s legit).

Signals?

Yes, signals. The type of Linuxy thing Julia Evans is great at teaching.

Signals are events that a process can listen for and choose to respond to. Hitting ctrl+c in your terminal, for example sends a SIGINT (interrupt) to the currently running process. This usually results in the process stopping.

You can also use the kill command to send a signal (any signal). Sending SIGINT via kill looks like this:

    # These are equivalent
kill -2 <process-id>`
kill -s INT <process-id>

  

Sending SIGKILL (kill -9 <process-id> or kill -s KILL <process-id>) is special - it will kill a process immediately. The process has no choice in the matter.

Let’s see how Laravel accomplishes graceful queue restarts, and how we can use that idea in our own code.

Laravel Queues

Laravel’s Queue library implement signals in order to stop workers gracefully. When a SIGTERM or SIGQUIT signal is received, the worker waits until the currently processing job finishes before actually stopping.

Therefore a job isn’t interrupted in middle of processing - it has the chance to finish.

This is done with a simple boolean variable. The Worker is basically just a while() {} loop. Every iteration it checks this variable, and stops if $shouldQuit == true.

We can see that Laravel listens for both SIGTERM and SIGQUIT signals here. Listening for these signals is setup just before starting the aforementioned while() loop.

Not too magical!

SIGINT

The terminate (SIGTERM) and quit (SIGQUIT) are both obviously named - they want a process to end (the difference is that SIGQUIT generates a core dump).

What about SIGINT (interrupt)?

This is the signal sent via ctrl+c. It’s typically only used in an interactive terminal - when we’re at our keyboard (for local development or for those 1-off tasks in production you really should automate).

You’ll notice that SIGINT isn’t listened for in Laravel’s queue worker! Instead, that’s handled by PHP, and it just quits whatever happens to be running. It is, therefore, not a way to gracefully exit a process!

Let’s see that quick. I created a job named LongJob that just sleeps for 10 seconds:

    <?php

namespace App\Jobs;

use ...

class LongJob implements ShouldQueue
{
    use ...

    public function handle(): void
    {
        Log::info("starting LongJob ".$this->job->getJobId());
        Sleep::for(10)->seconds();
        Log::info("finished LongJob ".$this->job->getJobId());
    }
}

  

I had a terminal open, running php artisan queue:work. Then I dispatched this job, and quickly hit ctrl+c. The logs showed that the job started, but never finished!

[2023-06-28 14:19:09] local.INFO: starting LongJob 1

If instead I send it signal SIGTERM, it will finish the job and then exit:

    # Start a worker
php artisan queue:work

# Find the process ID
ps aux | grep queue:work

# Dispatch a job, and then 
# kill the  worker with SIGTERM
# Process ID 69679 in my case
kill -s TERM 69679

  

We’ll see the job finishes before the process exits! However the sleep didn’t sleep for 10 seconds. More on that below!

[2023-06-28 14:19:09] local.INFO: starting LongJob 2  
[2023-06-28 14:19:11] local.INFO: finished LongJob 2

Graceful Restarts during Deployment

An in-production queue worker is typically monitored by a process monitor such as Supervisor. This restarts any process that stop in an unexpected way.

Laravel’s queue worker takes advantage of this by stopping the worker in a bunch of cases (error conditions, when artisan queue:restart is run, etc) as it can assume the queue worker will restart when needed.

Supervisor and friends usually stop a process by sending a SIGTERM signal, then waiting for the process to gracefully exit itself (which Laravel will do itself as describe above).

Typically, Supervisor (or whatever) will give the process a certain number of seconds to exit. If that time elapses, then SIGKILL is sent, and the process is forcefully stopped (since processes can’t ignore SIGKILL).

In Supervisor, the timeout is set by the stopwaitsecs options. The Laravel docs have this set to one hour (in seconds) in their example. You can lower this if your jobs don’t run a long time and wouldn’t ever need an hour to complete.

Using Signals in Our Code

Let’s see how to implement signals ourselves!

Jobs already complete fully when a signal comes in to stop them. From the perspective of us developers, we’re more likely to want to capture signals from within our artisan commands.

The first thing we’ll see is how a signal will stop a process (a running command) immediately.

Note that I’ll use “process” and “command” interchangeably here. Running a command like php artisan whatever spins up a PHP process. That process happens to run artisan, which boots the framework, runs our command, yadda yadda yadda. The point is: both words work here!

I created command LongCommand and had it sleep for 10 seconds (just like LongJob).

    public function handle()
{
    $this->info('starting long command: '.now()->format('H:i:s'));

    Sleep::for(10)->seconds();

    $this->info('finished long command: '.now()->format('H:i:s'));
}

  

If I run this via artisan longtime and use ctrl+c, it stops immediately:

php artisan longtime

starting long command 14:27:59
^C%

There was no output showing it finishing the command!

Trap

We can “trap” - listen for - a signal. This lets us run code before the command exits.

Trap lets you capture a signal, and do something in response, but it will then exit immediately.

This may actually be a bug, I’m not totally sure!

    public function handle()
{
    $this->trap(
      [SIGINT, SIGTERM], 
      fn($s) => $this->info('signal received: ' . $s)
    );

    $this->info('starting long command: '.now()->format('H:i:s'));

    Sleep::for(30)->seconds();

    $this->info('finished long command: '.now()->format('H:i:s'));
}

  

We trap SIGINT and SIGTERM and just echo out some information. This gives us a hook to run some cleanup code before exiting!

php artisan longtime

starting long command: 14:29:50
^Csigint received

The signal SIGINT was “trapped” but it still stopped the process! We get similar behavior for SIGTERM:

php artisan longtime

starting long command: 14:33:39
sigint received

So we can respond to a signal, but we can’t actually stop the process from exiting once the callback is run.

It would be useful if we could ignore the signal until we’re ready to exit! Luckily we can.

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Implementing SignalableCommandInterface

If our command implements Symfony’s SignalableCommandInterface, we can get the command to finish running before it exits.

(You can tell it’s a Symfony thing because it’s not named something pleasant like, say, Signalable).

On the surface, it looks like this works just like the $this->trap() method. However, if we return false in our handler, the command is able to finish its work.

Here’s what that looks like:

    # Some stuff omitted
use Symfony\Component\Console\Command\SignalableCommandInterface;

class LongCommandTwo extends Command implements SignalableCommandInterface
{
    public function handle()
    {
        $this->info('starting long2 cmd: '.now()->format('H:i:s'));

        Sleep::for(30)->seconds();

        $this->info('finished long2 cmd: '.now()->format('H:i:s'));
    }

    public function getSubscribedSignals(): array
    {
        return [SIGINT, SIGTERM];
    }

    public function handleSignal(int $signal)
    {
        $this->info('signal received: ' . $signal);
        return false;
    }
}

  

Since the signal handler returns false, our command is able to finish. How that works is just an implementation detail of Symfony’s handling of SignalableCommandInterface - it tells the code to not run exit($statusCode);.

After implementing that, we can see our “finished…” line is run:

php artisan long2

starting long2 cmd: 15:14:01
^Csignal received: 2
finished long2 cmd: 15:14:02

You might notice that we didn’t actually sleep for 30 seconds! This is specific to using sleep() for testing. Using signals actually shortcuts currently running sleep() calls, so if your code relies on that, it could be an issue!

Why do this?

A useful pattern for this is to do some cleanup work before exiting:

    # Some stuff omitted
use Symfony\Component\Console\Command\SignalableCommandInterface;

class LongCommandTwo extends Command implements SignalableCommandInterface
{
    protected $shouldExit = false;

    public function handle()
    {
        $this->info('starting long2 cmd: '.now()->format('H:i:s'));

        while(! $this->shouldExit) {
            $this->info("We're doing stuff");
            Sleep::for(1)->seconds();
        }

        // Pretend we're working hard on
        // cleaning everything up
        // Oh, also, this sleep actually happens
        // since it was started after the signal was received
        Sleep::for(10)->seconds();

        $this->info('finished long2 cmd: '.now()->format('H:i:s'));
    }

    public function getSubscribedSignals(): array
    {
        return [SIGINT, SIGTERM];
    }

    public function handleSignal(int $signal,)
    {
        $this->shouldExit = true;
        $this->info('Cleaning up: signal received: '.$signal);
        return false;
    }
}

  

You can now run some cleanup code either in the handleSignal() method or after the while() loop. This is great for deleting temporary files, ensuring data integrity when a command is turned off, closing networking connections, and all sorts of other stuff!

FilamentPHP: a first look

2023-06-26T00:00:00+00:00

The fastest way to get your admin panel up in the clouds? Deploy your Laravel application on Fly.io, you’ll be up and running in minutes!

As Laravel developers we know how productive a framework can be: It provides a set of common components, reusable functionality and most importantly, a set of guidelines that make everyone using the framework more efficient.

Knowing this, the fine folks over at FilamentPHP created a framework-within-a-framework, consisting of a couple of parts: There’s a Form Builder, a Table Builder and a Notifications package and there’s also an Admin Panel that combines them. Using the admin panel, you can create Resources that enable you to easily add CRUD pages with navigation, tables and forms to your app. It has all been expertly designed, and is very flexible to make your own changes if you need them. Check out their demo here: Filament Demo.

In this article, we’ll take a first look at Filament and we’ll build upon what we make now in future articles. Prepare for liftoff!

The app

In this series, I’m going meta.

No, not the social media/virtual reality company. Over the course of this mini-series, I’ll write an app that will show information about its own deployment on Fly.io. It’ll have a Machine model for every machine that’s in your app, with a relation to a Region model that specifies where that machine is running. This will enable us to do something very cool: we can let the machines ping each other and see how fast they respond. In this article, we’ll lay the foundations for that app.

Setting up Filament

So, create a new Laravel project and install the Filament admin panel with composer require filament/filament:"^2.0" . Now, run the migrations and create a user for yourself using php artisan make:filament-user .

Filament has been built to be an admin panel, but I’ll color outside of the lines and use Filament for the whole app. I’ve heard that Filament v3 will support this idea even better, but for now it also works well. To make it feel like Filament is the only thing in our app, I removed the default welcome page and changed the Filament URL from /admin to the / home url. You can do this by adding FILAMENT_PATH="/" to your .env file. After that, just remove the default route from web.php and delete the resources/views/welcome.blade.php file.

Now, onto the Filament admin panel itself. I’ll be using two models here: a Region model that has data about each region on Fly.io, and a Machine model that contains data about each machine where this app will be deployed. Machines will have a one-to-many relationship with Regions, so each Machine belongs to one Region. I’ll use this setup later on to illustrate the latency between different regions, but that’s for a different post 😉.

If you want to follow along, here are the models:

    class Region extends Model
{
    protected $fillable = ['iata_code', 'name'];

    public function machines(): \Illuminate\Database\Eloquent\Relations\hasMany
    {
        return $this->hasMany(Machine::class);
    }
}

  

Both iata_code and name are normal strings.

    class Machine extends Model
{
    protected $fillable = ['machine_id', 'region_id'];

    public function region(): \Illuminate\Database\Eloquent\Relations\BelongsTo
    {
        return $this->belongsTo(Region::class);
    }
}

  

Machineid is a string and regionid is a foreign Id for the Region model.

Adding the RegionResource

Let’s add a resource for our Region first, since that’s the easiest. The model has only two properties: the iata_code is the airport code of that region (LAX for Los Angeles) and the name is well, the name of the region.

Resources in Filament only describe how users interact with the Model, and how it should be presented, so make sure the Region model and database table already exist.

Resources in Filament are like superpowered classes that can handle all the CRUD operations using forms and tables. It’s almost like magic! You can pick if they need to show separate pages for viewing and editing models or you can opt for a ‘simple’ single view and use modals for creating and updating. I’m a simple person, so I’ll create a simple Resource for my Region model using php artisan make:filament-resource Region --simple . You’re probably smarter than me so you would let Filament auto-generate the forms and tables as well. Install the doctrine/dbal package using composer require doctrine/dbal --dev and then add --generate to the command: php artisan make:filament-resource Region --simple --generate. Much easier!

Here’s how it looks out of the box:

We actually have all we need already: we can do all the CRUD operations on our Region model, with validation on the form inputs! I’ll change the label for the IATA code to have IATA in all caps by opening RegionResource and adding ->label("IATA Code") on the TextInput for the iata_code property. While we’re at it we can also change the maxLength from 255 to 3. Here’s how the form() looks now:

    public static function form(Form $form): Form
    {
        return $form
            ->schema([
                Forms\Components\TextInput::make('iata_code')
                    ->label("IATA Code")
                    ->required()
                    ->maxLength(3),
                Forms\Components\TextInput::make('name')
                    ->required()
                    ->maxLength(255),
            ]);
    }

  

And we can do the same thing for the column in the table:

public static function table(Table $table): Table
    {
        return $table
            ->columns([
                Tables\Columns\TextColumn::make('iata_code')
+                   ->label('IATA Code'),
                ...
    }

This is how almost all customization on tables and forms will be done: create the correct field and use the static methods to change it to whatever you need!

One last thing: I pulled a sneaky on ya! I used a different icon for the ‘regions’ navigation item. I wanted something more region-like than the default ‘collection’ icon, so I changed the $navigationIcon on the RegionResource class from heroicon-o-collection to heroicon-o-globe. It just makes more sense.

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Adding the MachineResource

For the Machine resource, let’s switch it up a bit and not use the ‘simple’ resource. This means we’ll have different pages for index, edit, create and view operations. Run php artisan make:filament-resource Machine --generate et voilà: a MachineResource! For the Icon I’d suggest the heroicon-o-chip icon.

Let’s now take a look at the MachineResource. Something special happened here, and I need you to be as excited about it as I am: In the form method, there is a TextInput just like with the RegionResource, but now there’s also a Select. This lets you attach a related Region using the relationship on the Machine model. Filament will preload the Regions and let the user pick them from a list when creating a Machine. Pretty cool.

Let’s make it even cooler: let’s enable the user to filter the regions by typing in the field by adding ->searchable() and ->preload() on the Select field. The preload isn’t necessary here, but it does enhance the experience by showing results before the user has typed and filtering faster. Let’s see it in action!

Now, one final change to make before I let you return to watching cat videos: The MachineResource create form looks a bit weird, the input fields look like they’ve been thrown on the page. Let’s group them together in a card! I’ll also arrange both fields on one line, using ->columns(3) on the Card and ->columnSpan(2) on the Select field.

    public static function form(Form $form): Form
{
    return $form
        ->schema([
+           Forms\Components\Card::make()
+               ->schema([
                    Forms\Components\TextInput::make('machine_id')
                        ->required()
                        ->maxLength(255),
                    Forms\Components\Select::make('region_id')
                        ->relationship('region', 'name')
                        ->required()
                        ->searchable()
                        ->preload()
+                       ->columnSpan(2),
+               ])
        ]);
}

  

Here’s how it looks now:

Bear this in mind: we have only scratched the surface of what Filament can do, but we’ve already made a lot of progress: We already have CRUD operations for two models of our simple app, including form validation and relationships. On top of that we have authentication and navigation that’s built-in. Not bad, right? Later on we’ll customize the look and feel of our app, and after that we’ll add the dashboard that displays the ping times between the different deployments of our app.

And just between me and you: They don’t even know I’m writing this article. I just genuinely enjoyed using Filament and I couldn’t wait to tell you about it. Better yet, Filament v3 is right around the corner (check out v3when.com!) so they’ll probably have a lot more in store for us in the future. Keep it up guys!

Running Laravel Workers

2023-06-22T00:00:00+00:00

Need to run some Laravel workers? Deploy your Laravel application on Fly.io, you’ll be up and running in minutes!

We’re going to talk about running Laravel as a worker on Fly.io. This isn’t really anything special (we aren’t out to make things hard), but it’s useful to see how it essentially becomes a lesson in Docker.

The “usual” use case on Fly.io is running a web app, and then perhaps adding on some other services as needed. However, Fly.io is also great for just running background (or temporary) work loads, like queue workers!

So, here’s a few ways to run Laravel’s queue workers. Along the way we’ll pick up some tricks about Docker and Fly Machines.

Workers as Another Process

The “standard” way to run a worker is as we document it. It assumes you’re running your web application on Fly.io, and just want to add some queue workers.

To quickly do so, we can define a new process in our fly.toml file, and name it something useful, such as worker.

    [processes]
  app = ""
  worker = "php artisan queue:listen"

  

The app process is the standard web app (you always have an app process), while our new worker process will run the command given (artisan queue:listen).

We also need to adjust [http_service] section to ensure the HTTP health checks only apply to the app process:

    [http_service]
  internal_port = 8080
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0
+ processes = ["app"]

  

Note that defining additional processes will spin up a VM per process. In fact, for reliability, you’ll get 2 VMs! One is on standby, so you can pretend it’s just one VM for the most part.

How does the [processes] section work? It has to do with the Docker setup used to generate the Fly.io VM for Laravel apps.

Our Laravel Container

Docker uses ENTRYPOINT and CMD by concatenating them together to form the command used to run a Docker image (and therefore a Fly.io VM). For example, if we define an ENTRYPOINT as script /entrypoint.sh, and then a CMD as my-app, we’d end up running our VM with command /entrypoint.sh my-app.

In the standard Laravel container image we provide via fly launch, we define an ENTRYPOINT (but no CMD).

The ENTRYPOINT script is just some bash (/entrypoint.sh) that looks like this:

    #!/usr/bin/env bash

if [[ $# -gt 0 ]]; then
    exec "$@"
else
    exec supervisord -c /etc/supervisor/supervisord.conf
fi

  

If we pass it nothing (CMD is empty) it runs supervisord. If we pass it a different command (define a different CMD), it runs that command instead. Our default process app = "" runs Supervisor (which turns on php-fpm/nginx). Our process worker sets artisan queue:listen as the CMD and therefore runs that instead!

Workers Alongside Your App

What if we wanted a worker running in the same VM as our app, we can do that! This is a bit more traditional - like Laravel Forge’s setup - where everything is crammed into a single VM.

In this scenario, we can update Supervisor so it runs a worker as well as nginx/php-fpm. It takes a bit more work than defining a process, and also competes for resources in your Fly VM.

To do it, you can edit the files in the standard Docker setup you get via fly launch.

The base Docker image used is fideloper/fly-laravel. Inside the base image is Supervisor config found within /etc/supervisord.

The configuration for nginx/php-fpm (or octane, if you use it) will be found in /etc/supervisord/conf.d/(nginx.conf|fpm.conf). What you can do is add another file to the conf.d directory, with something similar to this:

    [program:worker]
priority=5
autostart=true
autorestart=true
stdout_events_enabled=true
stderr_events_enabled=true
command=php /var/www/html/artisan queue:listen
user=www-data
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

  

If you create this file in your codebase at .fly/worker.conf, and then update your Dockerfile, you can get this file to the correct place.

Note that I’m assuming you have a .fly directory, generated by the fly launch command. Here’s the updated Dockerfile that will incorporate the new worker.conf file:

    # Somewhere after this line...
COPY . /var/www/html

# Move the worker conf into conf.d to live with the
# other configurations
RUN mv .fly/worker.conf /etc/supervisor/conf.d/worker.conf

# Do the above before the line doing Nodejs stuff
FROM node:${NODE_VERSION} ...<and so on>

If you deploy your app with this setup, the worker.conf config will run artisan queue:listen in addition to the other standard configurations (php-fpm/nginx or octane).

We do NOT define an extra process in the [processes] section.

Just a Worker

If we wanted JUST a worker to be run, and never a web server, we could do a few things! One is to erase the other Supervisor configurations and replace it with our worker.conf from above. We’d have to adjust the Dockerfile like above, erase all other supervisor configurations, and finally remove the fly.toml‘s [http_service] section.

However I think the following is a bit less work - we can instead update our ENTRYPOINT script to never run Supervisor and always run a worker by default.

Here’s what it looks like - edit file .fly/entrypoint.sh and adjust it to run your worker instead of Supervisor:

    #!/usr/bin/env sh

# Run user scripts, if they exist
for f in /var/www/html/.fly/scripts/*.sh; do
    # Bail out this loop if any script exits with non-zero status code
    bash "$f" || break
done
chown -R www-data:www-data /var/www/html

if [ $# -gt 0 ]; then
    # If we passed a command, run it as root
    exec "$@"
else
    # Don't run supervisord by default
    # exec supervisord -c /etc/supervisor/supervisord.conf
    exec php /var/www/html/artisan queue:listen
fi

  

That’ll never run Supervisor and will only run whatever flavor of the queue:listen command you’d like. You can incorporate environment variables defined in your fly.toml as well:

    # If `QUEUE` is an env var in your fly.toml file
exec php /var/www/html/artisan queue:listen --queue $QUEUE

Any time you deploy this app, it’ll automatically run a worker now instead of the web app. Yet again, we do NOT define an extra process in the [processes] section.

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Automated Workers

You can run a Fly Machine directly (without using fly launch…fly deploy) via CLI or API calls. This is handy for automating workloads!

I outline how to do that in this article, where we run a custom artisan command. The only real difference for our use case here would be the command we define (artisan queue:listen... instead of artisan get-release).

The main trick is ensuring your app has a Docker image that Fly can use, either hosted publicly (Docker Hub, for example), or built and pushed up to the Fly registry before attempting to create a machine from that image.

Using the CLI, you can build the image on the fly - but through the API, you need to push the Docker image up to a registry ahead of time.

For CLI, it might look like this:

    fly apps create --name my-worker-app

# Assuming the app's Dockerfile and code
# base are in the current directory...
fly m run -a my-worker-app \
    --env "APP_ENV=production" \
    --env "LOG_CHANNEL=stderr" \
    --env "LOG_LEVEL=info" \
    --env "LOG_STDERR_FORMATTER=Monolog\\Formatter\\JsonFormatter" \
    . \
    "php" "artisan" "queue:listen"

  

Using the API requires that you build and push your Docker image to the Fly registry so the image is available. That looks like this:

    # Get your access token from ~/.fly/config.yml
# Or via `fly auth token`
export FLY_API_TOKEN="$(fly auth token)"

curl -X POST \
    -H "Authorization: Bearer ${FLY_API_TOKEN}" \
    -H "Content-Type: application/json" \
    "https://api.machines.dev/v1/apps" \
    -d '{
      "app_name": "my-worker-app",
      "org_slug": "personal"
}'

# Build the image locally. The image name must match
# app name we used when creating the machine app
docker build -t registry.fly.io/my-worker-app:latest

# Authenticate against Fly's registry
fly auth docker

# Push our newly tagged image
docker push registry.fly.io/my-worker-app:latest

# Create a machine using that Docker image
curl -X POST \
    -H "Authorization: Bearer ${FLY_API_TOKEN}" \
    -H "Content-Type: application/json" \
    "https://api.machines.dev/v1/apps/my-worker-app/machines" \
    -d '{
  "config": {
    "image": "registry.fly.io/my-worker-app:latest",
    "processes": [
      {
        "name": "do-some-work",
        "cmd": ["php", "artisan", "queue:listen"],
        "env": {
          "APP_ENV": "production",
          "LOG_CHANNEL": "stderr",
          "LOG_LEVEL": "info",
          "LOG_STDERR_FORMATTER": "Monolog\\Formatter\\JsonFormatter"
        }
      }
    ]
  }
}'

  

That’s more verbose and requires some extra work, but gives you an avenue to spin workers up dynamically (via code) if that’s what you need!

Laravel, React, and Inertia SSR on Fly.io

2023-06-15T00:00:00+00:00

In just a few steps, deploy your Laravel application on Fly.io, you’ll be up and running in minutes!

Need to run a Laravel backend along with a React frontend? That’s going to be a piece of cake with Inertia! Inertia allows us to work in both monolithic Laravel land, and our preferred (Inertia-supported) frontend framework or library.

What’s more, we can pre-render HTML pages on the server with Inertia’s Server Side Rendering support. This will help in improving initial page load of our Laravel-Inertia SPA app, and help our website become SEO-friendly.

And the cake on top? It’s super easy to set up on Fly.io!

Setting Up

The easiest way to set up a Laravel project with Inertia is through Laravel’s starter kits. Today, we’ll use the Breeze starter kit to quickly set up Laravel with React and Server Side Rendering.

Make sure you’re in your Laravel project’s directory, and run the following commands to set it up with Breeze and Inertia SSR for React:

# Get breeze
composer require laravel/breeze --dev

# Install with React and SSR 
php artisan breeze:install react --ssr

Running the last command above should scaffold necessary Breeze, React, and Inertia SSR settings into our project. But of course, we can still manually install Inertia SSR into our project by following instructions in the official docs.

With our Laravel app properly configured, we can now run php artisan serve for the backend, and npm run dev for the frontend. When we visit our site, we should see the default landing page for Breeze-scaffolded apps. And, if we right click on the page, click on “Inspect”, and navigate to the Console tab in the browser’s inspection window, we’ll see clear as day: React’s message on the console window:

Amazing! Thanks to Breeze’s Inertia Scaffolding, we now have a Laravel application scaffolded with Breeze’s authentication layer and React plugged in for its view layer—in just a few commands!

React

Let’s make sure we’re not dreaming, and edit this landing page, React style. If we would inspect the landing route declared in routes/web.php, we would notice that it returns this snippet:

    Route::get('/', function () {
    return Inertia::render('Welcome', [
        'canLogin' => Route::has('login'),
        'canRegister' => Route::has('register'),
        'laravelVersion' => Application::VERSION,
        'phpVersion' => PHP_VERSION,
    ]);
});

  

See, every page in an Inertia application would have a corresponding route, and JavaScript component. Controllers can be substituted with closures, just like above.

The “/” route renders a Welcome JavaScript component, which we can see full picture in resources/js/Pages/Welcome.jsx. Open it, and lo and be amazed— A full, “Welcome”, React component! In our Laravel app. That’s just, amaahy-ziing(!)—what else can be said? Except, maybe, nothing. Just appreciative silence(wonderment), for this React powered, Laravel welcome page:

Plugging a React Component

Moving away from basking at the brilliance of the experience above, let’s make our first change to our Laravel-React, Inertia bridged app.

Create a React component, resources/js/Page/Inspire.jsx, which will display inspiring quotes to visitors to our website:

    {/* resources/js/Page/Inspire.jsx */}

export default function Inspire( {quote} ) {
    return (
      <h1 className="flex justify-center"> 
         {quote}
      </h1>
    )
}

  

Plug this in the Welcome page, just under the Laravel logo:

    {/* resources/js/Page/Welcome.jsx */}

+ import Inspire from './Inspire';

export default function Welcome({ auth, laravelVersion, phpVersion }) {
  {/* Other snippet above the Logo here */}

  {/* Laravel Logo snippet here */}

+  <Inspire />

Notice however, our Inspire React function waits for a quote prop to be passed to it?

To handle this expectation, pass a $quote variable from our Laravel closure:

    /* routes/web.php */
Route::get('/', function () {
    return Inertia::render('Welcome', [
+       'quote' => strip_tags(\Illuminate\Foundation\Inspiring::quote()),
        'canLogin' => Route::has('login'),
        'canRegister' => Route::has('register'),
        'laravelVersion' => Application::VERSION,
        'phpVersion' => PHP_VERSION,
    ]);
});

  

And receive this quote props in both the Welcome and Inspire React components:

    {/* resources/js/Page/Welcome.jsx */}
+ export default function Welcome({ ..., quote }) {

    {/* Other snippet above inspire here */}

+   <Inspire quote={quote}/>

And so, with the Illuminate\Foundation\Inspiring package, and our plugged Inspire React component, our users may now view inspiring quotes on every visit:

Inertia SPA

Let’s rewind a little bit, step away from Laravel and React, and come back to Inertia. Huddle closer to the screen, right click somewhere on the current page, and click “View page source”. Then, search for “Laravel News” in the page source.

And notice, it’s…not there. Weird! There is definitely a “Laravel News” card in the welcome page if we go back to it—so, what is happening here?

To make sense of this sorcery, let’s look into Inertia’s protocol. See, first requests to an Inertia page would return an HTML document which contains a specific, data-page attribute. This contains a JSON encoded page-object, which Inertia uses to completely render the initial page content.

That’s right. Initially, our HTML page is not yet fully complete. Inertia by default relies on client-side rendering in showing our app’s pages.

This is the reason the page source we inspected earlier did not contain our expected string. The page source only contained partial HTML content sent from the server, along with Inertia’s trustworthy data-page metadata.

It was the client-side renderer in the browser, which is tasked into rendering the page’s content, that created the full content for us to see later on.

Inertia SSR

This initially incomplete content on our page can hurt our website’s SEO friendliness. Some search engines will only read the initial page content, while others would only wait a few seconds for remaining content to render before it leaves. This results in search engines missing content keywords useful for making our website more relevant in search engine results.

In order to improve our Laravel website’s SEO friendliness, Inertia provides us a means to instead render the full HTML content from the server, instead of the client. And this is through the use of its Inertia Server-Side Rendering support.

Earlier above, we did scaffold our application with the necessary command to enable Inertia SSR for our application. But, we missed one important step, which is running Inertia’s SSR server with:

php artisan inertia:start-ssr

Running the above command will start the Node process in our server. This SSR server will from then on handle fully rendering html content, that it will pass back to the client. So, notice now, once the SSR server starts running, we receive fully-complete, rendered HTML in our page source:

# Before starting SSR server
curl localhost:8000 | grep "Laravel News"
# Nothing found

# After starting SSR server
curl localhost:8000 | grep "Laravel News"
# We find some HTML

Flying on Fly.io

So, what are you waiting for? Run your Laravel React app with Inertia SSR—go on!

In fact, going from local to cloud with your Laravel-React, Inertia bridged app is a breeze in Fly.io. We even have a whole page for it in Fly.io. The steps are so easy, that we can even deploy our app, right here, right now!

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

First things first, install the flytcl command. Then access your account in your work environment either with a sign up with fly auth signup, or login fly auth login.

Next, in your Laravel project directory, run fly launch. Go through the prompts for details for your new Fly.io app ( app name, region to deploy, etc… ). But don’t deploy the app yet.

fly launch will scaffold files in your Laravel directory that will be used to deploy it on Fly.io. You’ll then need to make three general changes from there to successfully set up and deploy your app:

Revise the generated Dockerfile to include Node in the final image:

Inertia requires Node to run its SSR renderer, so revise the Dockerfile to include Node on the final image. Revise the Dockerfile to install Node under the fly-laravel image:

# Install node under fly-laravel image
+ RUN cd ~ \
+  && curl -sL https://deb.nodesource.com/setup_18.x -o nodesource_setup.sh \
+  && bash nodesource_setup.sh \ 
+  && apt install nodejs \ 
+  && cd /var/www/html

As you might notice however, there is already a section in the generated Dockerfile where a FROM node declaration is made. This From declaration is part of the Dockerfile’s multi-stage build setup in order to make use of Node in building static assets, but to not include Node in the final image created.

Since Node is needed in the final image to be created by the Dockerfile, and is included in the final image thanks to the revision above, you can now remove this multi-stage build set up:

    # REMOVE multi-stage build :
- FROM node:${NODE_VERSION} as node_modules_go_brrr
- RUN mkdir /app
- RUN mkdir -p  /app
- WORKDIR /app
- COPY . .
- COPY --from=base /var/www/html/vendor /app/vendor

# Also remove copying files between previously available Node image to final base image:
- FROM base
- COPY --from=node_modules_go_brrr /app/public /var/www/html/public-npm
- RUN rsync -ar /var/www/html/public-npm/ /var/www/html/public/ \
-     && rm -rf /var/www/html/public-npm \
-     && chown -R www-data:www-data /var/www/html/public

EXPOSE 8080

ENTRYPOINT ["/entrypoint"]

  

Revise fly.toml to create a new process group to run the Inertia SSR server, and include an [env] variable that points to its address:

With Node available in the final image generated from the Dockerfile above, you can now run Inertia’s Node-based SSR server. In this set up, you’ll create a new Fly process group which would be dedicated to running this SSR server.

To include a new process group in your Fly app, revise the configuration file generated, “fly.toml”, and include a new process group ( you can call this ssr ) which will run Inertia’s SSR server:
```
# Include ssr process group aside from default app process group:
+ [processes]
+   app=""
+   ssr="php /var/www/html/artisan inertia:start-ssr"

# Make sure to include a needed processes value for the section below:
[http_service]
  internal_port = 8080
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0
+  processes=["app"]
```
Creating a separate process group for your Inertia SSR server means running its own set of Fly Machine(s) separate from your Laravel web app’s set of Fly Machine(s). This means that you’ll need to set up communication between the Machine(s) running your Laravel web server, and the Machine(s) running your Inertia’s SSR server.

The Machine(s) running your Laravel web server can talk to the Machine(s) running your Inertia SSR server through .internal address. We can include this address in the fly.toml file by adding an [env] variable pointing to the ssr‘s .internal address, at its proper port:
```
[env]
  APP_ENV = "production"
  LOG_CHANNEL = "stderr"
  LOG_LEVEL = "info"
  LOG_STDERR_FORMATTER = "Monolog\\Formatter\\JsonFormatter"
  SESSION_DRIVER = "cookie"
  SESSION_SECURE_COOKIE = "true"
+  SSR_URL="ssr.process.<yourAppNameHerePlease>.internal:13714"
```
Revise config/inertia.php to point the ssr url to the SSR_URL variable configured:

With SSR’s .internal address available in your env, you can now use this to update the configured SSR url used by Laravel. Pull the config/inertia.php file into your Laravel directory, and revise the value for the ssr url:
```
'ssr' => [

    'enabled' => true,

    + 'url' => env('SSR_URL', 'http://127.0.0.1:13714'),
```
Once done with all the changes above, go on, and deploy your app with: fly deploy.

Great Job!

Truly, in this article, we saw how delightfully easy it is to bridge Laravel and React with Inertia—how much of a breeze it is to deploy with Fly.io.

And now, you should have your very own, SEO-friendly, Inertia-SSR powered( in a separate Machine from your ), Laravel-React app, flying on Fly.io—great job!

Laravel Queues with Other Languages

2023-06-06T00:00:00+00:00

Need to run some Laravel workers? Deploy your Laravel application on Fly.io, you’ll be up and running in minutes!

Learning multiple programming languages is good — there are a lot of benefits!

One such benefit is the ability to use what a language is strongest at. Some top-of-mind examples include Python’s various data-munging libraries, or Go’s concurrency tooling.

Now, you may want to use another language — but you likely aren’t about to rewrite your app. Luckily, I have some ideas! They look a bit like micro-services, but aren’t. Mostly.

What I like to do is sprinkle in other languages as part of my codebase’s “plumbing” — stuff that happens unseen, in the background.

One way to go about this is with gRPC. This lets you call a function, which (in the background) makes a network call to some other codebase, and gets a response. It looks like you’re making just any old function call, but it actually might be talking to another application. This is fun but gets complicated, relative to the 2nd idea.

That second idea: using queues! Queues are a tidy abstraction layer between languages because your codebase(s) can push or pull a job from a queue, and doesn’t need to care about who is reading or writing those jobs.

Let’s Do Some Queueing

Let’s see how to make use of Laravel’s queues with multiple languages.

There are 2 strategies you might employ:

Push to a queue from Laravel and read jobs in another language
Push to a queue from another language and read jobs in Laravel

The tricky bit is that Laravel creates (and consumes) jobs of a specific format. We need to know a job’s “shape” so we can read or write them correctly.

What’s In a Laravel Job

If you dispatch a job from Laravel into a queue, the payload looks something like this:

    {
    "uuid": "f69d79dc-2e0f-495a-9f70-62de0b4c7299",
    "displayName": "App\\Jobs\\SomeFancyJob",
    "job": "Illuminate\\Queue\\CallQueuedHandler@call",
    "maxTries": null,
    "maxExceptions": null,
    "failOnTimeout": false,
    "backoff": null,
    "timeout": null,
    "retryUntil": null,
    "data": {
        "commandName": "App\\Jobs\\SomeFancyJob",
        "command": "O:21:\"App\\Jobs\\SomeFancyJob\":1:{s:4:\"user\";O:45:\"Illuminate\\Contracts\\Database\\ModelIdentifier\":5:{s:5:\"class\";s:15:\"App\\Models\\User\";s:2:\"id\";i:1;s:9:\"relations\";a:0:{}s:10:\"connection\";s:6:\"sqlite\";s:15:\"collectionClass\";N;}}"
    }
}

  

The fun thing happening here is that the job key is specifying a class and method! In this case, it’s Illuminate\\Queue\\CallQueuedHandler@call. This is Laravel’s default handler - it does a bit of magic to rehydrate the job class (SomeFancyJob in our case) and run the handle() method on that. It’s a wrapper around our SomeFancyJob class.

That job key is the main bit — it defines what class and method Laravel should call when processing that job.

Finally, note that the command is serialized PHP - an instance of the SomeFancyJob class. More on that in a bit.

A Note on Queue Drivers

Unfortunately, we need to care about the queue driver. I tend to use Amazon’s SQS, since it’s the most legible — you can inspect jobs fairly easily and see the data within them.

Furthermore, SQS is API driven, which lets us avoid some thorny issues. Laravel does a lot of work for us in terms of ensuring we only pull one job at a time (redis, database, etc). With SQS, we’re just making calls to an API — SQS deals with the harder stuff. This means our “other language” code doesn’t need to care about locking mechanisms.

Processing Third-Party Jobs

Let’s see how to produce a job (from another language) that Laravel will later process via queue:work (or queue:listen).

We could mimic Laravel’s jobs ourselves and push a job that Laravel will handle without any further work. However, one thing I hate doing is asking another language to produce serialized PHP. It’s fragile! Luckily we aren’t forced to do that (although you still could, if you wanted).

What we can do instead is hook into the job key and have a class of our choosing handle processing the job! Our “other language” could add a job to the queue with a payload (a JSON string) like this:

    {
    "uuid": "f69d79dc-2e0f-495a-9f70-62de0b4c7299",
    "displayName": "App\\Jobs\\HandleThirdPartyJob",
    "job": "App\\Jobs\\HandleThirdPartyJob@call",
    "maxTries": 3,
    "maxExceptions": 1,
    "failOnTimeout": false,
    "backoff": null,
    "timeout": null,
    "retryUntil": null,
    "data": {
        "userId": 1,
        "foo": "bar"
    }
}

  

In this case, the class that will handle this message is App\Jobs\HandleThirdPartyJob (via its call() method).

The trade off here is that our HandleThirdPartyJob class needs to handle queue retries/failures (and so on) itself. We don’t get the benefit of the Illuminate\\Queue\\CallQueuedHandler@call class wrapping the job and doing much of that work for us.

A working example of HandleThirdPartyJob:

    <?php

namespace App\Jobs;

use App\Models\User;
use Illuminate\Queue\Jobs\SqsJob;
use Illuminate\Support\Facades\Log;

class HandleThirdPartyJob
{
    public function call(SqsJob $job, array $data): void
    {
        try {
            $u = User::findOrFail($data['userId'] ?? null);
        } catch(\Exception $e) {
            // Alternatively, to retry this job:
            // $job->release();
            return $job->delete();
        }

        Log::info('The user to perform stuff on is :' . $u->email);
        $job->delete();
    }
}

  

When to use this?

Your use cases will vary wildly. My use case has been for Chipper CI, where a little Golang service accepts streamed build output. For each “chunk” of build output, it creates an SQS (FIFO, in my case) message, which Laravel consumes - saving the build output and streaming it to the browser.

Using Golang to Push a Job

Here’s a bit of what it looks like for a Golang app to push a job into SQS in the format Laravel can read:

    // SendMessage takes req and converts it to an SQS job
// req is data (a Go struct) that can get turned into a JSON string
func SendMessage(req *SendMessageRequest) error {
  // Aws.Sess needs creating
  sqs := sqs.New(Aws.Sess)

  // Turn this data into a JSON string
  messageBody, err := json.Marshal(SqsMessageBody{
    DisplayName: "App\\Jobs\\HandleThirdPartyJob",
    Job:         "App\\Jobs\\HandleThirdPartyJob@call",
    MaxTries:    0,
    Timeout:     0,
    Data:        req, // data to turn into JSON
  })

  if err != nil {
    return fmt.Errorf("could not create job JSON for SQS: %w", err)
  }

  _, err = sqs.SendMessage(&sqs.SendMessageInput{
    MessageBody:  aws.String(string(messageBody)),
    QueueUrl:     aws.String(Aws.config.SqsUrl),
  })

  if err != nil {
    return fmt.Errorf("could not push message to SQS: %w", err)
  }

  return nil
}

  

Pushing to Third-Party Workers

This use case is a bit more interesting. What if Laravel pushed a job into a queue, and another language processed that job?

This is useful for all sorts of things — each language has their own strengths. Maybe you use Python to do some data munging (Python is really good at this). Maybe you use Golang to make use of concurrency while processing a job (sending lots of HTTP requests). There’s a lot of possible use cases!

Mohamed (who worked at Laravel) recently wrote up an excellent example of this.

What’s this look like in Laravel? Well, it turns out that Laravel can push a bunch of ad-hoc data into a queue:

    # These both work
Queue::push(['foo' => 'bar']);
Queue::push("nonsense", ['foo' => 'bar']);

  

And the respective payloads from those calls look like this:

    {
  "uuid": "8c7c2034-05e4-4321-8a69-d892d6f6703d",
  "displayName": null,
  "job": {
    "foo": "bar"
  },
  "maxTries": null,
  "maxExceptions": null,
  "failOnTimeout": false,
  "backoff": null,
  "timeout": null,
  "data": ""
}

{
  "uuid": "ade43fc5-c530-4bb3-b6ba-b0fbc6cf6245",
  "displayName": "nonsense",
  "job": "nonsense",
  "maxTries": null,
  "maxExceptions": null,
  "failOnTimeout": false,
  "backoff": null,
  "timeout": null,
  "data": {
    "foo": "bar"
  }
}

  

Since whatever language we choose to process the job won’t care about the Laravel-specific data, we can decide if we like our relevant data being within job or data keys (or even both).

Here we can receive said message. Note that the SQS message.Body property is a string — the JSON string that we’ve been talking about above.

    // Here, we expect the json "data" key to 
// be in a Nonsense struct, which
// in turn expects to be created from
// JSON in format {"foo": "some-string"}
type LaravelMessage struct {
  Uuid string `json:"uuid"`
  Data Nonsense `json:"data"`
}

type Nonsense struct {
  Foo string `json:"foo"`
}

func ReceiveMessage() (*Nonsense, error) {
  // Aws.Sess needs creating
  sqs := sqs.New(Aws.Sess)

  // Get one message at a time
  msgResult, err := sqs.ReceiveMessage(&sqs.ReceiveMessageInput{
    QueueUrl:            someQueueURL,
    MaxNumberOfMessages: aws.Int64(1), // one message
    VisibilityTimeout:   someTimeout,
  })

  if len(msgResult.Messages) > 0 {
    msg := msgResult.Messages[0]

    // Convert the SQS message body into the Go struct
    // defined above, which expects a Laravel-formatted job.
    // Note: message.Body is the JSON string containing
    // uuid, data, and other Laravel job data
    laravelMsg := &LaravelMessage{}
    err := json.Unmarshal([]byte(*message.Body), laravelMsg)
    if err != nil {
      return nil, fmt.Errorf("could not parse SQS message body: %v", err)
    }

    return laravelMsg, nil
  }

  return nil, fmt.Errorf("no sqs message found in queue: %s", someQueueUrl)
}

  

Golang, being strictly typed (and being Golang), is a bit verbose because of how it needs you to predefine what data you expect to get in a JSON string.

All we’re doing here is grabbing the SQS message Body and converting it to an object. We start by assuming the message body is a Laravel-generated JSON string. Then we grab just the bits we care about (the uuid and the data). Our Nonsense struct definition lines up with the data found in the JSON string, and so Golang is able to create an object from the JSON.

After that, our Go program can do whatever it needs with the data!

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Other Queue Drivers

As mentioned, I like using SQS because pushing and popping jobs (messages) off the queues just involve some API work — gory details of “at most once” or “at least once” guarantees are taken care of for you.

For other drivers, we need to care about how we push/pop jobs to the queue storage.

Pushing a job into the queue for other drivers isn’t too hard — depending on the driver (sqs, redis, database, etc).

Redis push method is here (it’s a LUA script)
Database push method is here (it’s just an insert!)

Popping (pulling/getting) a job gets a bit trickier, as you need to care about the driver’s locking mechanisms (to ensure multiple workers don’t consume a job more than once).

Redis pop method is here (again, a bit of LUA)
Database pop’s relevant method is here
- It uses some locking, but the exact locking mechanism changes per database driver via the getLockForPopping() method

SQS is the easiest there. Redis is probably okay-ish (just steal some LUA from the Laravel framework core), and Database is the most annoying, since the syntax and locking mechanism changes per database type.

It’s totally doable to use these techniques with other drivers, it’s just a bit more annoying — API work is generally the easiest (and SQS is cheap).

Improving Client Side Pagination with Livewire

2023-06-01T00:00:00+00:00

Need a quick Laravel app on the cloud? Deploy your Laravel application on Fly.io, you’ll be up and running in minutes!

The bane of client side pagination stems from our retrieval of entire bulky data sets in one go. But, do we really need to get the entire data set all at once?

In this article, we’ll apply a combination of data allowance and accumulation strategies to remove this “all data at once” bottleneck and improve client side pagination. We’ll do so quickly and easily, with the help of Livewire!

The Problem

Let’s imagine we have a table called “UserDetails”. It contains user addresses, phone numbers, and life-long mottos. We want to display these records in the right amount of rows that visibly makes sense. So we display it in a client paginated table, 10 rows per page. To our (anticipated) horror, the number of records grew, and so did the time it took to render the table:
Client Side Pagination has always been associated with retrieving entire data sets in one go. Once the entire dataset is available in the client, pagination can simply be done on that stored data, removing calls to the server to display the next or previous page.

The client paginated table above was built on the same principle, where it initially waits for the entire UserDetails to get downloaded before it could render the table. So, as the size of its data grew, so did the time it took to retrieve it.

The Solution

How about, instead of getting the entire data set in one go, we get it in parts?

We can apply client side pagination to an initial subset of data stored in the client, provided the subset contains enough data to allow initial pagination—let’s say, three pages worth of data ( Data Allowance ). Then, in the background, we silently call for more data allowance to add on top of this existing subset, allowing us to eventually complete the entire data set we want to paginate( Data Accumulation )!

Set Up

Livewire provides an easy bridge of communication between PHP and JavaScript. It will allow us to easily merge existing subset of table rows in the client with remaining subsets in the server. So, let’s start by creating a Livewire component for our table: php artisan make:livewire PaginatedTable.

Afterwards, in its component class, declare and initialize a $headers property to list headers in the table, and a $rowCount for the number of rows to show per page:

    /* app/Http/Livewire/PaginatedTable.php */
class PaginatedTable extends Component
{
  public $headers = [];
  public $rowCount;

  public function mount(){
    // An array of visible fields of model
    $this->headers  = UserDetails::tableHeaders();

    // Number of rows per page
    $this->rowCount = 10;
  }

  

Data Allowance

Next, let’s create a method setNextBatchData() to retrieve our data subset. It will get the necessary number of rows per page via $rowCount, then add in an extra pinch of “allowance” by getting two more pages worth of data:

In order to share this subset with the client, we create a data-updated browser event through Livewire’s dispatchBrowserEvent.

We’ll later setup our view’s JavaScript to listen to this event and update our table’s content with new subsets.

    /* app/Http/Livewire/PaginatedTable.php  */
public function setNextBatchData(){
    $data = UserDetails::limit( $this->rowCount*3 )->get();
    $this->dispatchBrowserEvent('data-updated', ['data'=>$data]);
}

  

If we show 10 rows per page, setNextBatchData() will retrieve 10*3 rows instead. Giving us the first page, plus two additional pages of data to work with.

Of course, we can always adjust this allowance. It can be increased or decreased such that the initial data we send back is enough for multiple pages in the client, but light enough to not cause a bottleneck in page loading.

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Rendering The Table

Next, let’s set up a table in our view. We’ll set up its header row with $headers from our Livewire component, and also include a <nav> section for our Next and Prev buttons:

    {!-- resources/views/livewire/paginated-table.blade.php --!}
<table>
    <thead>
        @foreach( $headers as $header )
        <th>{{ $header }}</th>
        @endforeach
    </thead>
    <tbody id="tBody">
    </tbody>
</table> 

<nav>
  <button onclick="prevPage()">Prev</button>
  <button onclick="nextPage()">Next</button>
</nav>

  

Once our HTML’s good to go, we set up JavaScript to render data received from the data-updated event declared above. To start, declare variables we’ll be using for paginating:

Notice Livewire’s @js directive in the script? We use it to easily convert PHP variables to their respective JavaScript counter parts:

$headers => mHeaders
$rowCount => rowCount

    <script>
// Data for our table to use
let mData    = [];
let mHeaders = @js( $headers );

// Pagination reference to display mData items in table
let page     = 1;
let startRow = 0;
let rowCount = @js( $rowCount );
let tBody = document.getElementById("tbody");

  

Then, we get initial pages of our table by making a call to the setNextBatchData() method in the server. This method will dispatch a data-updated browser event we can listen to in order to merge new data into the mData array:

Livewire provides the @this directive to easily make a request to methods in a Livewire component. This is only callable once Livewire has loaded.

    // Make a request to the setNextBatchData method in the server
document.addEventListener('livewire:load', function () {
    @this.setNextBatchData();
});

// Merge incoming subset data into client's  array of rows, `mData`
window.addEventListener('data-updated', event => {
    mData = [...mData, ...event.detail.data];
    renderPage();
});

  

Every time we get a new subset from the data-updated event, we re-render the table’s visible rows with renderPage(). This will clear the <tbody> tag, then loop through mData to insert rows into the table. It’ll use startRow to get an item from mData for the page’s first row, and rowCount to determine how many more rows to display after:

    function renderPage(){
    // Clear the content of the table
    tBody.innerHTML = '';

    // Add items starting from the `startRow` index 
    for(
        let row=startRow; row<mData.length && row<startRow+rowCount; row++
    ){
        // Insert an item into a row
        let item = mData[row];
        var rowTable = tBody.insertRow(-1);

        // Show item's attributes in row's cells
        for( let header of mHeaders ){
            var cell = rowTable.insertCell(-1);
            cell.innerHTML = '<div>'+ item[header]+ '</div>';
        }
    }
}

  

Client Side Pagination

Above we initially show the first page only. Let’s now add pagination elements into our view so users can move forward and backwards across our table’s pages. Since we rely on startRow to get the nth item from mData to start a page, let’s calculate this first:

    /* resources/views/livewire/paginated-table.blade.php  */
function getStartRow( page ){
  return (page*rowCount)-rowCount;
}

  

Then add in a nextPage() function. It will first check if the next page’s starting row is within the bounds of the mData array. If so, it will increment page, and update the startRow. Finally it will call the renderPage() method to re-render the <tbody>‘s content:

    function nextPage(){
  var newStartRow = getStartRow( page+1 );
  if( newStartRow < mData.length ){
      page = page+1;
      startRow = newStartRow;
      renderPage();
  }
}

  

Then create another function prevPage()to allow our users to move to a previous page. This movement is only possible if the current page is above page 1. If so, page is decremented, a new startRow is set, and the table content is re-rendered:

    function prevPage(){
    if( page > 1 ){
        page = page-1;
        startRow = getStartRow( page );
        renderPage();
    }
}

  

Since our data in the client ends at page 3, nextPage() won’t get past that page. It’s time to add more data allowance into our client with—Data Accumulation!

Data Accumulation

To accumulate data, we simply add more data on top of an existing subset of data. This means we’ll have to merge more data into the mData array in our client. We can get more data through our component’s setNextBatchData(). Let’s revise it to retrieve rows only after the last batch’s last item. To do so, we’ll have to keep a reference, $lastId, to keep track of the last item from the previous subset:

    /* app/Http/Livewire/PaginatedTable */
+ public $lastId;

public function mount(){
    $this->headers  = UserDetails::tableHeaders();
    $this->rowCount = 10;
+   $this->lastId   = 0;
}

  

We revise setNextBatchData() to get rows after this id. Once the new subset is retrieved, we update the $lastId to the current subset’s last item:

    public function setNextBatchData(){
// Get Rows after id
+   $data = UserDetails::where( 'id', '>', $this->lastId )
    ->limit( $this->rowCount*3 )
    ->orderBy('id','asc')
    ->get();

// Update last id
+   if( $data && $data->last() ) 
      $this->lastId = $data->last()->id;  

  

Now that we have setNextBatchData() set up to get the next subsets for our table, the next question is, when do we actually call it to get more data?

In a previous article, we used Livewire’s polling mechanism to periodically get data from the server, and add it on top of the data stored in our client. But this time, we’re going to be thrifty in our network calls ( and data consumption! ), and instead only call additional data whenever the user moves to the next page.

So, we call setNextBatchData() in our client JavaScript’s nextPage() function:

    /* resources/views/livewire/paginated-table.blade.php  */

function nextPage()
{   
    // Next page logic here...
    // Get more next page allowance
    @this.setNextBatchData();
}

  

Every time we click on the Next Page button, the table will be re-rendered to display the page requested. Then, at the end, a background call is sent to setNextBatchData() so that we’ll still have more “data allowance” to keep our client pagination happy.

Merging more data into mData will increase the rows stored in our client until we complete the entire dataset. We can track the number of total rows vs the rows stored in our client:

    /* resources/views/livewire/paginated-table.blade.php */
window.addEventListener('data-updated', event => {
    mData = [...mData, ...event.detail.newData];
    renderPage();

// Display row tally
+   document.getElementById("curRows").innerHTML = 'Current Rows: '+mData.length;
+   document.getElementById("totRows").innerHTML = 'Total Rows: '+event.detail.totalCount;
});

  

And with that, we have our all new, lightweight, client side paginated table, no longer bogged down by the heaviness of an entire table:

Some Things to Take Notice Of

Now there are still some elephants that need to be addressed in the room:

Eventually the entire dataset is going to get downloaded into the client’s browser, and might still be a bit heavy. So as much as possible, we’d want to “reset” the data accumulated every now and then. Perhaps, reset the accumulated data whenever none-pagination table interaction occurs. Like say, during Search, filter, or even sort?
Notice how there’s just Prev and Next pages in our navigation. Can we not have other pages here? Of course we can! We can easily show the page numbers we currently have data for. What about pages not yet in our client side? Well that will take a bit more steps and a mix of server side pagination. We’ll need to get the requested page+allowance in the server, and insert those rows in the proper indices in mData.

Lastly, please notice, with this improved client side pagination, we now have: a lag-free movement between pages, and a less heavier, definitely lighter, client paginated table.

Full Stack Laravel

2023-05-30T00:00:00+00:00

Fly runs apps close to users by taking Docker images and transmogrifying them into micro-vms, running across the globe. You can ship a Laravel app on Fly.io in just a couple of minutes.

You can use fly launch to get you up and running quickly with Laravel, but let’s talk about all the other stuff you probably want:

Redis
MySQL
cron (Laravel Scheduler)
Queues

We’re gonna spin up a new Laravel installation, add some user authentication (backed by MySQL), setup Laravel’s scheduler, and then see how to use Redis for queues, session storage, and caching.

The Laravel Application

The first thing we need is a Laravel application. We can use fly launch to get one running really quickly.

    # 1. Create a shiny, new Laravel app
composer create-project laravel/laravel laravel-fly
cd laravel-fly

# 2. Launch on Fly!
fly launch
# Set app name, select region, deploy now

# 3. View the site in your browser:
fly open

Redis

Rather than trying to run everything in one VM on Fly, extra “services” are best run as separate VMs.

You can create your own Redis instance on Fly, but you might find it easier to use the managed Redis by Upstash, which is just a few commands away.

If you do want to roll your own, you can follow the Laravel + Redis docs.

    # A new, empty directory
mkdir redis && cd redis


# Create a new redis app (don't deploy it yet!)
fly launch --image flyio/redis:6.2.6 --no-deploy --name lara-redis


# Add a persistent volume so Redis data sticks around
# Note: Volumes only live within the region they are created
fly volumes create redis_server --size 1


# Update fly.toml config to use the volume
cat >> fly.toml <<TOML
[[mounts]]
  destination = "/data"
  source = "redis_server"
TOML


# Add a password (before launching!)
fly secrets set REDIS_PASSWORD=somepassword

  

A few things to note!

We created a Volume. In conjunction with a tweak to fly.toml, using persistent storage makes sure data isn’t lost when the app is re-deployed (however, volumes are specific to a single region).
We gave Redis a password (by setting a secret), we didn’t leave it open for just any old connection. This is required.
As documented, don’t forget to edit the generated fly.toml file to delete the [http_service] section, OR the [[services]] section, if present.

After that, we’re ready to deploy it.

fly deploy

Private Networking

We’ll want our application to talk to Redis. Luckily, private networks are a thing in Fly.

We named our Redis service lara-redis, which means other services (within our Fly organization) can reach the Redis service using hostname lara-redis.internal.

Laravel Config

We need to update the Laravel configuration to use Redis. Laravel needs to know the hostname and password of our Redis service.

Passwords are secrets, so we’ll set it as such in our app service.

    cd ~/Fly/laravel-fly

# Add in our redis password secret to this app as well
fly secrets set REDIS_PASSWORD=somepassword

Adding a secret triggers a quick re-deploy. Neat!

The secret is set, but we also need additional (not-so-secret) configuration to tell Laravel where and how use Redis. Update the Laravel app’s fly.toml file:

[env]
  APP_ENV = "production"
  CACHE_DRIVER = "redis"
  SESSION_DRIVER = "redis"
  REDIS_HOST = "lara-redis.internal"

Let’s test that this works! If we edit file routes/web.php, we can add a route that uses Redis:

    Route::get('/test', function() {
    Cache::put('new-key', Str::random(16));

    return "The new cached string is " . Cache::get('new-key');
});

We updated some files, we need to deploy again to get that change in place:

fly deploy

You should be able to head to your-app.fly.dev/test to see the result. If you don’t get any errors, then everything is working!

MySQL

Fly offers to set up an HA cluster of PostgreSQL for you.

There’s no need to brag about it, but we know that MySQL is pretty great. For that, we can use PlanetScale in our Fly apps.

That link shows how to sign up and create a MySQL database on PlanetScale. Once created, we just need to add the proper configuration into our application.

You can also run MySQL as “just another Fly.io app” as well.

Edit fly.toml and update the environment variables:

[env]
  APP_ENV = "production"
  CACHE_DRIVER = "redis"
  SESSION_DRIVER = "redis"
  REDIS_HOST = "lara-redis.internal"

+  DB_CONNECTION = "mysql"
+  DB_HOST = "some_hostname.us-east-2.psdb.cloud"
+  DB_DATABASE= "laravel-fly"
+  MYSQL_ATTR_SSL_CA = "/etc/ssl/certs/ca-certificates.crt"

Note the addition of MYSQL_ATTR_SSL_CA=/etc/ssl/certs/ca-certificates.crt, which is required for connecting to databases on PlanetScale (Using TLS for remote database connections is extremely important)!

The value /etc/ssl/certs/ca-certificates.crt is correct for the container Fly setup for you via fly launch. See here for other possible values.

Laravel needs to know the username/password for the database as well. Those seem pretty secret, so let’s use secrets!

I opted to keep the hostname an environment variable, but you might want to set it as a secret.

    fly secrets set DB_USERNAME=your-username DB_PASSWORD=your-password

  

Authentication with Laravel Breeze

Many Laravel apps will have user authentication. Let’s use Laravel Breeze to quickly scaffold auth into our app.

    cd ~/Fly/laravel-fly

composer require laravel/breeze --dev

php artisan breeze:install

# If you're running your app locally
# you'll want to get static assets/run migrations
npm install
npm run dev

php artisan migrate

Check that it works locally if you’d like, then deploy it!

fly deploy

Did you know you can ssh into your apps? We’ll use that to run our migrations:

fly ssh console -C "php /var/www/html/artisan migrate --force"

Connecting to top1.nearest.of.black-rain-2336.internal... complete
Migration table created successfully.
Migrating: 2014_10_12_000000_create_users_table
Migrated:  2014_10_12_000000_create_users_table (216.52ms)
Migrating: 2014_10_12_100000_create_password_resets_table
Migrated:  2014_10_12_100000_create_password_resets_table (199.27ms)
Migrating: 2019_08_19_000000_create_failed_jobs_table
Migrated:  2019_08_19_000000_create_failed_jobs_table (197.86ms)
Migrating: 2019_12_14_000001_create_personal_access_tokens_table
Migrated:  2019_12_14_000001_create_personal_access_tokens_table (310.91ms)

Migrate on Deploy

If you’re wondering how to automate the running of your migrations, you’re in luck! That’s configurable in the deploy section of the fly.toml file.

The following will run your migrations just before making the latest version of the application available.

    # File: fly.toml
[deploy]
  release_command = "php /var/www/html/artisan migrate --force"

  

Release commands run your latest code in their own VM just before making the actual application VM live. However the release command has no access to any attached volumes (we’re not using any volumes in our case here).

You should now be able to register and log in.

It works, we’ve got a full(ish) Laravel app running!

Cron and Queues

The fly launch command gave us some boilerplate to easily use Laravel’s scheduler (cron) and queue workers.

To enable those, we can configure some processes in our fly.toml file. Add the following:

    [processes]
  # Yes, this should be an empty string
  app = ""
  cron = "cron -f"
  worker = "php artisan queue:listen"

  

You may need to do further configuration to your queue worker (setting the driver).

Don’t forget to run fly deploy after making changes.

Be sure to check the docs on Laravel’s cron and queues to see more details, like how to scale them out separately. Lastly, note that each defined process runs in a separate VM.

Poke the Bear

Let’s see what’s going on in our VM. We can use the logs command to see what’s going on behind the scenes, or the ssh command to poke around!

    # From our laravel app directory
# (where fly.toml is)
cd ~/Fly/laravel-fly

# See some logs!
fly logs

# SSH and interact with the VM!
fly ssh console

# We can see that our php-fpm, queue, and cron daemons are running:
> ps aux | grep [f]pm
> ps aux | grep [q]ueue
> ps aux | grep [c]ron

  

What We Did

We have a Laravel application running with all the usual things:

MySQL
Redis
cron (for the scheduler)
Queues

That covers the most popular use cases for Laravel, but there’s lots more you can do as well! My favorite part of this is how easy it is to setup extra services in Fly (Redis in our case). The private networking makes it super easy.

Fly is working on making database integrations easier as well. Keep an eye out for managed MySQL via PlanetScale. You can see how to take your app global with Planetscale here.

Parsing Recipes with Robot Help

2023-05-23T00:00:00+00:00

Fly can run your Laravel apps, workers, and more around the world. Want to spin up a Laravel application? You’ll be up and running in minutes!

This is NOT a story about how Javascript stole 2 hours of my life, and PHP rescued me - in 6 minutes. That did happen, but I digress.

The real story is: Recipe sites are bloated. Rumor has it that this is because you can’t copyright a recipe, but can copyright the content surrounding a recipe.

In any case, I wanted to see how hard it was to parse recipes and just get the important parts. I had dreams of tokenizing HTML strings and creating a ton of conditionals (I think we used to call that AI) to remove the flowery cruft.

It’s actually a lot easier than that - thanks to SEO.

SEO to the Rescue

Due to Google’s insistence, almost all recipe sites embed standardized Recipe information somewhere in the HTML — most often in the form of a <script type="application/ld+json"> tag that contains some JSON. This JSON is fashioned after whatever schema.org dictates for Recipes.

Still, it’s a bit tricky to parse JSON, and took a tad of experimentation.

The basic process is to get some HTML from a recipe, and do some XML parsing to find the bits that matter for Recipe parsing. Then run that through a parser to get the bits you want.

This is all a breeze in PHP, which I mention only because the HTTP client used along with the most common ld+json parser in Node couldn’t return HTML from a site that returned gzipped content. In 2023. Listen, I’m not angry. It just sounds a lot like it.

Parsing Recipe Data

A composer package made this pretty easy: https://github.com/brick/structured-data. This package could read/parse HTML and find any schema data within it. Our job then is to find Recipe-specific data, and grab what we want.

Here’s the full code base - it has more details than what I’ll mention here.

    use App\RecipeParser;
use Brick\StructuredData\Reader\MicrodataReader;
use Brick\StructuredData\Reader\RdfaLiteReader;
use Brick\StructuredData\Reader\JsonLdReader;
use Brick\StructuredData\HTMLReader;

$parsers = [
    new JsonLdReader(),
    new MicrodataReader(),
    new RdfaLiteReader(),
];

$url = request('recipe');

$response = Http::throw()
            ->get($url);

// UTF8 isn't handled correctly otherwise
$html = mb_convert_encoding($response->body(), 'HTML-ENTITIES', "UTF-8");

$recipe = null;
foreach($parsers as $parser) {
    $reader = new HTMLReader($parser);
    $items = $htmlJsonReader->read($html, $url);

    if($recipe = RecipeParser::fromItems($items, $url)) {
        break;
    }
}

// If we didn't get a recipe after
// our 3 different reader attempts...
if (! $recipe) {
    // failure
}

// Do something with the valid recipe
return $recipe;

  

Then we need our RecipeParser class, which takes the $items and finds Recipe data within, if available.

The JSON (and thus our $items) is in a format I found to be a bit convoluted, but it’s manageable. The hardest part was running the code through a ton of recipe sites and accounting for variations in how they output Recipe schemas.

Our RecipeParser class is a bit mundane, it mostly involves these 2 methods - fromItems and parse.

The static method fromItems() is the “entrypoint” into the class - it takes the $items (retrieved from the meta data parser) and the $url. It tries to find a Recipe schema within the given data.

If we find a recipe, the parse() method will grab each “section” of a recipe, and see if there’s a corresponding method to parse it out.

    // A static function to serve as the entrypoint to the parser
public static function fromItems($items, $url)
{
    foreach($items as $item) {
        // Normalize schema "type", find any that contain "recipe"
        // We often get a Recipe mixed in with other data types (e.g. "article")
        $types = Str::lower(implode(',', $item->getTypes()));
        if(Str::contains($types, 'recipe')) {
            return (new static(url: $url))->parse($item);
        }
    }

    // Sometimes you don't get a "type" defined, but the
    // entire JSON document might be a recipe
    if (count($items) == 1) {
        return (new static(url: $url))->parse($item);
    }

    // No recipe data type found
    return false;
}

// This does the "hard" work of parsing out a recipe
public function parse(Item $item): Recipe
{
    // For each "recipe" property, we check if the class has a related method
    // e.g. `public function parse_ingredientlist() {}`
    foreach($item->getProperties() as $name => $values) {
        $fn = "parse_".Str::replace(['http://schema.org/', 'https://schema.org/'], '', Str::lower($name));
        if(method_exists($this, $fn)) {
            $this->$fn($values);
        }
    }

    return new Recipe(
        $this->title, $this->url, $this->author, 
        $this->ingredients, $this->steps, $this->yield, 
        $this->totalTime, $this->images
    );
}

  

There’s a bunch of data munging to account for differences you see across the various sites in how they show the Recipe metadata.

For example, one method is to get the name of the recipe:

    protected function parse_name($values)
{
    $this->name = (is_array($values) ? $values[0] : $values);
}

  

The variables $values is usually an array, but if $values is not, then we just assume it’s a string. Others might have multiple values, or be instances of Brick\StructuredData\Item, in which case we need to do extra work to parse the Item classes (essentially we need to recursively parse Item classes).

Fly.io ❤️ Laravel

Need a place in the clouds to fly your app? Deploy your servers close to your users with Fly.io, it’ll be up in minutes!

Deploy your Laravel app! →

Sprinkle in AI

If we can’t find relevant metadata in a recipe site, maybe we can use some AI to help us! Frankly, we could probably use AI for every request, but given current pricing and how fast (slow) the OpenAI API is, using this as a fallback makes sense to me.

Thanks to SEO and recipes being a whole industry, most recipes you’ll find on the first few pages of Google results are optimized, and so you don’t need AI to parse them.

However, to add AI, all you need is PHP OpenAI package. It abstracts OpenAI’s various API’s really nicely.

In our case, we use the Chat API, as it allows us to use the 3.5-turbo model. I don’t yet have GPT 4 access!

What I did was add in a 4th “Reader” class that talks to OpenAI:

    <?php

namespace App;

use OpenAI;
use DOMDocument;
use App\Recipe;
use Brick\StructuredData\Reader\JsonLdReader;
use Illuminate\Support\Str;

class AIRecipeReader
{
    public static function read($url)
    {
        $prompt = "Extract only the following information from the recipe found here: $url

            - dishName
            - publishDate (in YYYY-MM-DD format)
            - total cook time (in human-readable format)
            - author
            - ingredients 
            - steps (array of strings)
            - servings

            Please generate the output as valid JSON, preferably in ld+json format based on schema.org specification.";

        $client = OpenAI::client(config('ai.open_ai_key'));

        $result = $client->chat()->create([
            'model' => 'gpt-3.5-turbo',
            'messages' => [
                ['role' => 'user', 'content' => $prompt]
            ],
        ]);

        $dom = new DOMDocument;
        $html = mb_convert_encoding('<script type="application/ld+json">'.$result->choices[0]->message->content.'</script>', 'HTML-ENTITIES', "UTF-8");
        $dom->loadHTML($html);
        $items = (new JsonLdReader)->read($dom, $url);
        return RecipeParser::fromItems($items, $url);
    }
}

  

Implementing this is just a matter of adding in a conditional if a Recipe object isn’t found.

The fun part here is the prompt. We asked OpenAI to return valid ld+json, and we (usually) get that back! We just feed that into the JsonLdReader class.

That class needs an instance of DOMDocument, so we hack that into place before parse the result. We also did our usual UTF-8 encoding to ensure we get the correct characters when needed.

Here’s a thing to note tho: OpenAI’s API responses are pretty slow! You’ll know you hit OpenAI when the request to get a recipe back feels like it’s taking forever (~10-20 seconds).

You can check out the results at https://recipeplz.fly.dev!

Console Applications with Laravel Zero

2023-05-17T00:00:00+00:00

Need a quick Laravel app on the cloud? Deploy your Laravel application on Fly.io, you’ll be up and running in minutes!

Here’s a news flash: You can write console applications with Laravel! Actually, it’s kinda with Laravel and kinda without. Let’s take a closer look at Laravel Zero and some cool things you might want to know about!

Support for Facades

First off: there’s support for Facades, like Process, Http, or Log. This enables us to fire off api calls with one command. Here’s an example from an upcoming package that shows how to get a list of the organizations you’ve made on Fly.io:

Firstly, we’ll need to authenticate, just like flyctl would. To do this we can ask flyctl to print the token it uses by running fly auth token . Create a new command with php application make:command ExampleCommand and add this in the handle() method:

    public function handle()
{
    try
    {
        $result = Process::run('fly auth token')->throw();
        $token = $result->output();
        $this->info("got token successfully");
    }
    catch (ProcessFailedException $e)
    {
        $this->error($e->result->errorOutput());
        return Command::FAILURE;
    }

    return Command::SUCCESS;
}

  

I’ve wrapped everything in a try-catch block so whenever a Process::run method fails, it throws an error (the →throw() method after Process::run() takes care of that) and that error gets caught, prints out what went wrong and makes sure to return Command:FAILURE. This way we can jsut assume everything went smoothly, and if not the catch block will pick it up! This beats having to write an if-else block for every Process:run to check if it was successful or not, and print out the error if it wasn’t.

Now that we have the auth token let’s do something with it, shall we? We can get all your organizations on Fly.io like this:

    try
{
    $result = Process::run('fly auth token')->throw();
    $token = $result->output();
    $this->info("got token successfully");

+   $response = Http::withToken($token)
+       ->acceptJson()
+       ->contentType("application/json")
+       ->post('https://api.fly.io/graphql', ["query" => "query {currentUser {email} organizations {nodes{id slug name type viewerRole}}}"]);
+
+   // organizations will be an array of arrays that look like this: array("id" => , "slug" => , "name" => , "type" => , "viewerRole" => )
+   $organizations = $response->collect("data.organizations.nodes")->toArray();
+   $this->info("get organizations successfully");
}

  

Notice you can use the HTTP facade, just like you’re already used to in Laravel? It’ll be convenient like this all the way, which is just 👨‍🍳💋! You just have to install it before using it, or you’ll get a BindingResolutionException. Just run php <your-app-name> app:install http and you’re set!

Creating tasks

A wise man once said : “Console applications are only as good as their output looks”. Okay, that’s probably completely false, but still… Let’s compare the outputs of our example app with how they could look:

The bottom one looks better, right? This is done by defining tasks. Each tasks has a title and an anonymous function. “loading…” is displayed when the function is executing, and it updates to a ✔ when the function returns true or to a ❌ when it returns false. Sweet, right?

These anonymous functions can introduce scoping issues if you’re not careful, though! By default, anonymous functions in PHP do not share scope with the parent where they’re defined. This means that we can’t use variables that are defined outside of the anonymous function. Luckily, there’s the use keyword which is very useful (get it?): it can share variables from the outside into the anonymous function. There are two ways to mitigate this:

use ($variable): the value of $variable can be updated, but the parent scope won’t inherit the changes made to it inside the anonymous function. Let me make this extra clear:

    $variable = 1; // value is 1
$this->task("Scoping example", function () use($variable) {
    $this->info($variable); // will print '1' to console output
    $variable = 23; // value becomes 23
    $this->info($variable); // will print '23' to console output
}); // changes made to $variable will be forgotten here
$this->info($variable); // this will print '1' to console output

  

So basically what use($variable) does is copy over the value from the parent’s $variable into a new $variable inside the function. The changes inside the function aren’t copied over!

use(&$variable) : the value of $variable will be shared between the parent and the anonymous function. This is what you want to use if you want to keep the changes done to $variable inside the anonymous function. Let’s compare this with use($variable):

    $variable = 1; // value is 1
$this->task("Scoping example", function () use(&$variable) { // notice the '&' added here
    $this->info($variable); // will print '1' to console output
    $variable = 23; // value becomes 23
    $this->info($variable); // will print '23' to console output
  }); // changes made to $variable will NOT be forgotten here
  $this->info($variable); // this will print '23' to console output, instead of '1'!

  

This is the difference by passing by value like use ($variable)) and passing by reference like use(&$variable).

Here’s how the tasks look when accounting for possible scoping issues:

    + $token = "";
+ $this->task("Getting token", function() use (&$token) {
    $result = Process::run('fly auth token')->throw();
    $token = $result->output();
+ });

+ $organizations = [];
+ $this->task("Getting organizations", function () use($token, &$organizations) {
    $response = Http::withToken($token)
        ->acceptJson()
        ->contentType("application/json")
        ->post('https://api.fly.io/graphql', ["query" => "query {currentUser {email} organizations {nodes{id slug name type viewerRole}}}"]);

    // organizations will be an array of arrays that look like this: array("id" => , "slug" => , "name" => , "type" => , "viewerRole" => )
    $organizations = $response->collect("data.organizations.nodes")->toArray();
+ });

  

Fly.io ❤️ Laravel

Need a place in the clouds to fly your app? Deploy your servers close to your users with Fly.io, it’ll be up in minutes!

Deploy your Laravel app! →

Creating interactive menus

There are a whole host of cool features to show, but I’ll wrap it up here with interactive menus. Let’s say I was creating a package to deploy apps on Fly.io with just one command (I am) and I needed to ask the user in what organization they want to deploy in (I need to). Then I’d get all the organizations like I showed above (I do) and show them in a menu for you to select from (I will). Here’s how I did exactly that:

    +   $selectedOrg = $this->menu("Select Organization", array_column($organizations, 'name'))
+       ->open();
+
+   if ($selectedOrg =="")
+   {
+       // user selected "Exit"
+       $this->info("Exiting.");
+   }
+   else
+   {
+       // user selected an organization
+       $this->info("Selected Org Name: ");
+       $this->info($organizations[$selectedOrg]['name']);
+   }
}
    catch (ProcessFailedException $e)

  

This creates a menu in the command line (!) that looks like this (!!!) :

But the craziness doesn’t stop there, oh no! There’s a whole bunch of customization possible here. I can make that same menu look like this:

Here’s how:

        $selectedOrg = $this->menu("Select Organization", array_column($organizations, 'name'))
+       ->setForegroundColour('magenta')
+       ->setBackgroundColour('white')
+       ->setWidth(200)
+       ->setPadding(10)
+       ->setMargin(5)
+       ->setExitButtonText("Abort")
+       // remove exit button with
+       // ->disableDefaultItems()
+       ->setTitleSeparator('*-')
+       ->addLineBreak('<3', 2)
        ->open();

    if ($selectedOrg =="")
    {
        // user selected "Exit"
        $this->info("Exiting.");
    }
    else
    {
        // user selected an organization
        $this->info("Selected Org Name: ");
        $this->info($organizations[$selectedOrg]['name']);
    }
}
    catch (ProcessFailedException $e)

  

Packaging it up

Those were just some of the possibilities of Laravel Zero. I’ll leave you with some quick notes about building the console application: with php <application-name> run:build <build-name> you can build a standalone application. Do note that it requires PHP to be installed on the client.

Another way to use the commands you’ve built is by using composer to require your package. I didn’t want to dirty Packagist with my in-development package, so here’s how I added it using github instead:

    // add this in the lowest level in composer.json, on the same level as 'name', 'license' and 'require'.
"repositories": [
    {
        "type": "vcs",
        "url": "https://github.com/<organization name here>/<repo name here>"
    }
],

  

Then, you can run composer require <organization name>/<repo name> to install the latest release on the github project.

I’ll stop typing now. I hope this has been useful, if there’s ever some small to medium size task you want to automate, think about Laravel Zero!

Creating a Middleware to Globally Log Submissions in Livewire

2023-05-16T00:00:00+00:00

Need a quick Laravel app on the cloud? Deploy your Laravel application on Fly.io, you’ll be up and running in minutes!

Ever implemented logic that applied to multiple Livewire components? Why not use a middleware to make it available across Livewire components?

For example: Imagine having two Livewire components that handle two different form submissions. If, say, each of these components handle their form submissions through a submit() method, how can we easily log the submissions they receive?

We might do something like log the submission details in the submit() method of each component, right before processing the submission. But imagine having more than two components of —say three, five, or more? That’s going to be a ton of repetitive logging.

In this post, instead of repeating logic in each component’s submit() method, we’ll apply our logic in a single middleware to make it available for Livewire components that trigger a submit() method.

Version Notice. This article focuses on Livewire v2, details for Livewire v3 would be different though!

Let’s start

To start, let’s set up a simple Livewire component that handles a form submission. It will take in a name and email, and save it to the database.

Create the component with php artisan make:livewire ContactForm and add the following view:

    {{-- resources/views/livewire/contact-form.blade.php --}}
<div>
  <form wire:submit.prevent="submit">
    <input type="text" wire:model.defer="name">
    @error('name') <span class="error">{{ $message }}</span> @enderror

    <input type="text" wire:model.defer="email">
    @error('email') <span class="error">{{ $message }}</span> @enderror

    <button type="submit">Save Contact</button>
  </form>
</div>

  

The form is wired to the submit() method, while each of the text input field are wire:model.defer-ed to their respective public properties in the component:

    /* app\Http\Livewire\ContactForm.php */

class ContactForm extends Component
{
  public $name;
  public $email;

  public function submit()
  {
    \App\Models\Contact::create([
      'name' => $this->name,
      'email' => $this->email
    ]);
  }

}

  

The only time a request is made to the Livewire component’s route is during form submission, which calls the component’s submit() method. This is how it looks like:

The request is sent to livewire/message/contact-form, and we can see three items in its updates payload. One syncInput type for each wired text input field, containing values for name and email, and finally a callMethod type to trigger a call to the submit() method of the component.

Logging Requests for Submit Method

Let’s say, we want to log submission details made through the submit() method of our Livewire component. Create a middleware with php artisan make:middleware LogInputRequests:

    /* App\Http\Middleware\LogInputRequests */
use Log;

class LogInputRequests
{
  public function handle(Request $request, Closure $next): Response
  {
      // 1. Make sure Livewire's updates key exists
      if( isset($request->updates) ){
        foreach( $request->updates as $update ){   

            // 2.Find target "submit" method
            if( 
                isset($update['payload']['method']) && 
                $update['payload']['method'] == 'submit' 
            ){
                $userId = $request->user()? $request->user()->id : 'NA';

                // 3. Log Stuff
                Log::info( 
                  "User: $userId, Path: $request->path(), 
                  Submission:". json_encode($request->updates) 
                );
                break;
            }

        }
      }

      // Proceed
      return $next($request);
  }
}

  

The above middleware processes requests specific to Livewire components. Of course, since it’s handling requests to Livewire components, we have to be a bit mindful in creating this middleware. When creating middlewares that handle Livewire component requests, we generally have to answer two questions:

The first question is, when should this middleware be applied? Livewire relies on configured middleware groups that’s applied on all Livewire components. This means we’ll have to specify conditions that will check whether to apply the middleware or not. Otherwise, it will run its logic for all Livewire components. And, do we want that?—no, not in this use case.

So, notice above, we check for the existence of a submit method in the request’s updates key. This key is Livewire’s magical bag that let’s it know about changes in the view that the server needs to sync, or trigger methods with. We check for the submit method because we want to log submission details, which are processed through this method. And, notice, we do not limit this to one specific component. Rather, it is applicable for any, so long as the submit method was triggered. This allows us to log submission for not just our component above, but for other components triggering a submit() method as well.

The second question is, what part of the Livewire request do we include in the intended logic? In our case, we want to log submission details. Submission details would of course be found in the updates bag, so we simply log the updates bag which will contain values for name and email, as well as the call to the submit() method. Of course, we can further parse this to our liking or use case.

And speaking of parsing—please remember: We’re logging user input here. Make sure, and do not forget, to sanitize this.

Fly.io ❤️ Laravel

Need a place in the clouds to fly your app? Deploy your servers close to your users with Fly.io, it’ll be up in minutes!

Deploy your Laravel app! →

Registering the Middleware in Livewire config

Livewire’s config file config/livewire.php allows us to register middleware groups in its middleware_group attribute.

In order to apply our middleware logger to requests made on our Livewire components, we create a new middleware group in app/Http/Kernel.php, where we include our middleware class:

    /* app/Http/Kernel.php */

protected $middlewareGroups = [
  // other middleware groups
  'for_livewire'=>[
      \App\Http\Middleware\LogInputRequests::class,
  ]

  

Then, we pull Livewire’s config file into our project with php artisan livewire:publish --config, and include our recently declared middleware group here:

    /* config/livewire.php */
'middleware_group' => ['web','for_livewire'],

Every request made to a Livewire component via its route livewire/message/ will now pass through our middleware. Of course, thanks to our logic of checking for the submit method, only requests containing the submit method in the updates key will be logged.

So now, if we click on the submit button of our Livewire component, the submission details should now be logged, likeso:

And, just like that—with one middleware—we have our instant submission logger, globally available to any Livewire component—neat!

Conclusion

There are many other use cases for implementing logic into middleware, just like applying policies to safeguard access to your Livewire components.

Moving logic into middleware makes sense when that logic is repeated in multiple, different places. What’s more, Livewire automatically applies one Livewire-valid middleware to all Livewire components, making application of Livewire-wide logic a convenient task to do with middlewares.

Do you have any other Livewire-wide logic to implement, if so, why not implement it into a middleware?

Securing Access to Livewire components with Inline Policies, Traits, and Middlewares

2023-05-04T00:00:00+00:00

Deploy now on Fly.io, and get your Laravel app running in a jiffy!

How do we safeguard Livewire components against unauthorized access? How do we restrict access to its view or specific action based on a current user’s permissions?

Laravel Policies

In this article we’ll go through three different ways to restrict access to Livewire components with the help of Laravel Policies:

Applying policies with authorization or gates - to restrict access to an entire component view
Applying policies within traits - to reuse the same restriction across different components, either for the entire view or specific action
Applying policies through middleware - to restrict access to a Livewire component’s specific action

Version Notice. This article focuses on Livewire v2, details for Livewire v3 would be different though!

Set up

To start, we’ll create a “ViewButton” Livewire component with php artisan make:livewire view-button. This component will be responsible for two things: 1️⃣ Open a User’s profile page in a new tab, and 2️⃣Increment the number of times there was an attempt to visit the User’s profile.

The component’s view will contain a button which will open a new tab with the User’s profile page, and call an incrementAttempt() method in the component on click:

    {{-- resources/views/livewire/view-button.blade.php --}}
<div>
  <a href="{{ url('users/'.$userId) }}" target="_blank">
    <button wire:click="incrementAttempt">View</button>
  </a>
</div>

  

In the component, we’ll declare the public attribute $userId to determine the user selected for viewing, and the method incrementAttempt() which will increment the number of times there was an attempt to view the User’s profile:

    /* app/Http/Livewire/ViewButton.php */

use App\Models\User;

class ViewButton extends Component
{
    public $userId;

    public function incrementAttempt()
    {
          $user = User::find($this->userId);
          $user->increment('view_attempt');
    }
}

  

Where do we get the $userId? We pass it down from a parent component, likeso:

    @livewire('view-button', ['userId' => $user->id])

  

With our component set up, let’s put some restrictions on it 🔒

Creating Our Policy

Let’s say, we don’t want just anyone to be able to view a user’s profile. We want to restrict access to only the user itself, or another user with either an Administrator or Auditor role. Let’s create a Laravel Policy for this, run php artisan make:policy UserPolicy.

This will generate a file for us at \app\Policies\UserPolicy. In here we can create different methods representing different authorization checks, which we’ll use to restrict access to our Livewire component.

Restricting Views with A Policy

Create a policy method that checks if a current user has permission to view a user’s profile. It will take in two parameters, $loggedInUser and $userToCheck. This will only “authorize” if the $loggedInUser is either an Administrator, an Auditor, or has the same id as the $userToCheck:

Notice the UserLevel class in the code? We’re using enums here to make our user roles readable.

    /* app\Policies\UserPolicy */
use App\Model\User;
use App\Enums\UserLevel;

class UserPolicy
{
  public function view( User $loggedInUser, User $userToCheck )
  {
    return $loggedInUser !== null && 
      ( 
        $loggedInUser->level == UserLevel::Administrator ||
        $loggedInUser->level == UserLevel::Auditor ||
        $loggedInUser->id == $userToCheck->id
      );
  } 

  

Laravel auto detects policies based on the model name. The policy above, UserPolicy will get matched to the User model and will automatically be made available to use through Laravel’s authorizes helper. However, if the policy name does not match with any model, make sure it is registered, otherwise, it will not be detectable in authorize helper.

Applying Policies with Authorization or Gates

With our view policy setup above, let’s use it to restrict access to our ViewButton component’s view. One way to apply it is to use Laravel’s authorize() helper before we render the view of the component:

    /* app/Http/Livewire/ViewButton.php */

+ use Illuminate\Foundation\Auth\Access\AuthorizesRequests;
+ use App\Models\User;

class ViewButton extends Component
{
+   use AuthorizesRequests;
    public $userId;

    public function incrementAttempt(){}

    public function render()
    {
+       $userToCheck = User::find($this->userId);
+       $this->authorize( 'view', auth()->user(), $userToCheck );
        return view('livewire.view-button');
    }

  

We pass our policy method’s name view to the authorize helper, along with the parameters the policy method takes in. If the policy method returns false, the authorize method will throw an Illuminate\Auth\Access\AuthorizationException exception:

Hmmm. But. We want to still see other parts of our parent component. We just want to hide the ViewButton component. Let’s catch this exception, and return an empty element instead:

    /* app/Http/Livewire/ViewButton.php */

public function render()
{
    try {
        $userToCheck = User::find($this->userId);
        $this->authorize( 'view', auth()->user(), $userToCheck );
        return view('livewire.view-button');
    } catch (\Illuminate\Auth\Access\AuthorizationException $e) {
        return '<div></div>';
    }
}

  

Of course, we can replace this try catch with Gates instead:

    /* app/Http/Livewire/ViewButton.php */

use Illuminate\Support\Facades\Gate;
use App\Models\User;

class ViewButton extends Component
{
    public $userId;

    public function render()
    {
        $userToCheck = User::find($this->userId);
        if( Gate::allows( 'view', auth()->user(), $userToCheck ) ){
          return view('livewire.view-button');
        }else{
          return '<div></div>';
        }
    }

  

Now, we can have our parent component and everything else with it, with the exception of the ViewButton component, unscathed:

Applying Policies within Traits

We might want to apply the same policy to another component. To avoid duplicating the work we just did above, we can enclose the policy check into a trait, and declare this trait in any Livewire component we want to restrict on the same policy!

Create a trait at app\Traits\WithViewAuthorization.php:

    /* app\Traits\WithViewAuthorization.php */

use Illuminate\Support\Facades\Gate;
use App\Models\User;

trait WithViewAuthorization
{
    /* Override th render method of any Livewire Component*/
    public function render()
    {
        $userToCheck = User::find($this->userId);
        if( Gate::allows('view', auth()->user(), $userToCheck) ){
            return view( $this->viewPath );
        }else{
            return '<div></div>';
        }
    }
}

  

Notice how we declared a render() method in our trait above? We’ll use this to render a Livewire component’s view instead of its normal render method. It should also be able to use the public attributes of our Livewire component, just like the $userId, and a new attribute, $viewPath( to determine our blade path! ).

Back in our ViewButton component, let’s revise it to:1️⃣ apply the trait above, 2️⃣ set the $viewPath value to the proper blade path, and 3️⃣ remove the render() method:

    /* app/Http/Livewire/ViewButton.php */

+use App\Traits\WithViewAuthorization;

class ViewButton extends Component
{
+   use WithViewAuthorization;

+   public $viewPath = 'livewire.view-button';
    public $userId;

-   public function render(){}
}

  

Alright! Save the changes, refresh the page, and marvel at the use of traits 🪄

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Restricting Actions with A Policy

By hijacking the render method of our Livewire component, we’re able to restrict access to our component’s view. But. Let’s say, we want to make sure a specific action from our component is only made available based on another policy?

Let’s re-imagine our incrementAttempt() logic. Maybe. we’d want to only increment if the current user has an Auditor role? Let’s create another policy in our UserPolicy:

    /* app\Policies\UserPolicy.php */
public function increment( User $loggedInUser )
{
    return $loggedInUser !== null && 
      $loggedInUser->level == UserLevel::Auditor;
} 

  

Then, back in our WithViewAuthorization trait, let’s add a new method called allowIncrement() that will check the increment policy above:

    /* app\Traits\WithViewAuthorization.php */

public function allowIncrement()
{
  $userToCheck = User::find($this->userId);
  return Gate::allows('increment', auth()->user(), $userToCheck );
}

  

Since our Livewire component ViewButton is using the above trait, it can call the new method above in any of its own action methods:

    /* app/Http/Livewire/ViewButton.php */

public function incrementAttempt()
{
  if( $this->allowIncrement() ){
    // Increment user profile view count
  }else{
    // Handle unauthorized increment attempt, 
    // maybe log this, or just ignore it
  }
}

  

Applying Policies through Middleware

Aside from applying policies directly or through traits, we can also apply them through middleware. First, make sure to remove the allowIncrement() check above. We’ll be applying policy checks through middleware, so we’ll no longer need it.

Once done, create a middleware to apply our increment policy above, run: php artisan make:middleware EnsureIncrementAllowed.

To apply this middleware to the incrementAttempt() in our Livewire component, we’ll have to: 1️⃣ Add it to a middleware group that’s included in Livewire’s middleware_group config. Create a new middleware group, and include our middleware there:

    /* app/Http/Kernel.php */
protected $middlewareGroups = [
  'policies' => [
    \App\Http\Middleware\EnsureIncrementAllowed::class,
  ]

  

2️⃣ Pull the livewire config file, then include our new middleware group policies:

    /* config/livewire.php */

'middleware_group' => ['web','policies'],

Once this middleware group is included in Livewire’s config, all subsequent requests to any Livewire components will pass through this middleware group. But, we only want to apply this middleware to ViewButton‘s call to its incrementAttempt() action.

To do so, we’ll have to specify in our middleware logic to apply itself only to the route used by Livewire’s request to ViewButton—“livewire/message/user-view-button”—and make sure the action being called is incrementAttempt():

    /* app/Http/Middleware/EnsureIncrementAllowed.php */

class EnsureIncrementAllowed
{
    public function handle(Request $request, Closure $next): Response
    {
      // Make sure we're applying to proper route & action
      if( 
        $request->path() == 'livewire/message/user-view-button' && 
        isset($request->updates[0]['payload']['method']) && 
        $request->updates[0]['payload']['method'] == 'incrementAttempt' 
      ){
        // Apply policy check
        if( !Gate::allows('increment', auth()->user() ) ){
          return response()->json([
              'effects'=>['html'=>null, 'dirty'=>[], 'dispatches'=>[]],
              'serverMemo'=>['checkSum'=>$request->serverMemo['checksum']]
          ]);
        }
      }
    }
}

  

Notice above how we respond back with effects and serverMemo data? This is because Livewire expects a response with these data. In fact, we can even go so far as to dispatch our own custom browser event from here! Let’s say incrementNotAllowed:

    if( !Gate::allows('increment', auth()->user() ) ){
  return response()->json([
      'effects'=>['html'=>null,'dirty'=>[],
        'dispatches'=>[
          ['event'=>'incrementNotAllowed','data'=>[]]
        ]
      ],
      'serverMemo'=>['checkSum'=>$request->serverMemo['checksum']]
  ]);
}

  

Which we can listen for in our livewire view:

    {{-- resources/views/livewire/view-button.blade.php --}}
<script>
    window.addEventListener('incrementNotAllowed', event => {
        console.log('Did not increment view, policy not passed!');
    })
</script>

  

Of course, this method is a bit inconvenient, specially with the need to check for the route being used, and the action being called. But, it’s a good start to applying policies through middleware.

Summary

In this article we went through three different ways of restricting access to a Livewire component’s view and its action.

The first being the application of in-line policy checks with authorizes() and Gates provided by Laravel. The second by transferring the in-line authorization checks into a trait, includable in our Livewire component. And finally, the last through checks from a middleware( from which we even managed to send back a browser event! ).

Of course, these are not the only ways to restrict access to Livewire components. There are tons more! Like calling a @can() blade helper before rendering a component, or directly applying checks before triggering an action.

There’re many ways to safeguard access to a Livewire component, applying “policy gates” is just one way!

Prometheus Metrics in Laravel

2023-05-01T00:00:00+00:00

You can send prometheus metrics to Fly.io - for free. Spin up a globally-running Laravel app in minutes!

It’s almost a hidden feature - Fly.io provides Grafana dashboards with helpful metrics for all of your Fly.io apps.

All you have to do is head to https://fly-metrics.net/.

👉 Here’s the kicker: If your app publishes some metrics, Fly.io will read them. You can then view them in the provided Grafana dashboard! 🪄

The metrics just need to be Prometheus-flavored.

Instrument your App

We’re going to talk about how to “instrument” your app. Here’s a super dry definition:

The term instrumentation refers to an ability to monitor or measure the level of a product’s performance and to diagnose errors.

So, we’re going to “instrument” a Laravel application by having it generate Prometheus-friendly metrics. Fly.io will read those and let us graph them in the hosted Grafana at fly-metrics.net.

Since Fly.io provides the infrastructure to get this done, all we need to care about is our code base.

Grafana + Prometheus

An extremely popular pairing is Prometheus (for collecting metrics) and Grafana (for graphing metrics).

Prometheus works by “pulling” metrics into it - it’s basically a web scraper, reaching out with HTTP requests to various configured locations to find metrics.

If your application creates an endpoint that spits out Prometheus-friendly metrics, than Fly.io can consume them and make them visible for use in the Grafana dashboard!

This means baking a URI endpoint, such as /metrics, into your application for Prometheus to read.

Fly.io actually uses Victoria Metrics, which is compatible with Prometheus.

We’re going to see how to generate metrics and expose them for Prometheus. These ideas are applicable to apps in any language or framework.

Metric Types

There are a few different metric types you can create:

Counters
Gauges
Histograms
Summaries

Counters are values that only go up. They can’t go down, but they can be reset to zero.

These are useful when you want to see the rate of occurrences of something over time. Usually the actual count is less interesting than the rate they occur, or the rate of change. A common example is the count of inbound HTTP API requests made to your application.

Gauges can go up and down in value, and are perhaps the most commonly used metric. An example Gauge value is server RAM or CPU usage.

Histograms will count occurrences of events (or whatever your data is) into buckets. Percentile charts with the classic bell-curve distribution is a histogram. You might use this to be able to say “90% of HTTP responses were 2-3k in size”.

Summaries are like histograms, but it can calculate quantiles over a sliding time window. Does that sound like I know what I’m talking about? You’ll have to read about them yourself.

What to Measure

Let’s see how to instrument our Laravel applications. We’ll tell certain parts of our code to record metrics when something interesting happens.

What counts as “interesting” depends on your application.

I just did this work for Chipper CI, so I’ll use examples from that! This is an app that runs continuous integration builds for Laravel apps.

I wanted metrics on the following things:

Number of builds over time - Fairly simple, a count of builds over time. This never goes down, and I’m interested to see trends in this number (and be able to break them down per team)
Number of paid builds - This one is interesting as it starts at zero at the beginning of each month (the billing period), and goes up depending on the plan a team is paying for. This number goes up when customers go over their free tier (or if they go over builds they pre-paid for).

I chose a counter for the first, and a gauge for the second.

The majority of the work here is just figuring out our Prometheus metrics. The code side of things isn’t so hard!

These metrics are both business metrics and (sort of) ops metrics! That’s fine, I like measuring whatever is useful.

Metrics in Laravel

There’s a few things we need to do in order to get these metrics collected and visible to Prometheus:

Prometheus only scrapes for metrics periodically, so we need a way to persist our metrics.
We need to provide a /metrics (or similar) endpoint that spits out metrics.
Our code needs to collect the metrics.

We’ll use the PromPHP Prometheus Client which is actively maintained. There are a few Laravel wrappers for this out there, but they seem unmaintained (apologies to any open source maintainer of those if I’m getting that wrong).

Since writing this article, Spatie has released package Laravel-Prometheus, which does some of this work for you!

Persisting Metrics

I persisted my metrics in Redis. The PHP package provides Redis as one of it’s storage mechanisms - we just need to tell it where our Redis instance is.

I used a Service Provider for this:

# Oh yeah, and install the library
composer require promphp/prometheus_client_php

# Then make a service provider
php artisan make:provider PrometheusServiceProvider

And in that service provider:

    <?php

namespace App\Providers;

use Prometheus\Storage\Redis;
use Prometheus\CollectorRegistry;

use Illuminate\Support\Arr;
use Illuminate\Support\ServiceProvider;
use Illuminate\Contracts\Foundation\Application;

class PrometheusServiceProvider extends ServiceProvider
{
    public function register()
    {
        $this->app->singleton(CollectorRegistry::class, function() {
            Redis::setDefaultOptions(
                Arr::only(
                     config('database.redis.default'), 
                     ['host', 'port', 'password']
                )
            );

            return CollectorRegistry::getDefault();
        });
    }
}

  

We tell Laravel to create a singleton of class CollectorRegistry (the main entrypoint of the library). Whenever we request that class, we want Laravel’s container to send back the same instance of CollectorRegistry that we create above We take the opportunity to set its Redis storage to use our default connection information.

Be sure to register the service provider in your config/app.php file!

Note: If you use Upstash Redis on Fly.io, you’ll get a connection URL for Redis. It will look like redis://default:<some-password>@<some-hostname>. You can break this into pieces for Laravel’s REDIS_HOST and REDIS_PASSWORD fields (ignore the “default” username).

A Facade

This singleton we registered with Laravel’s container is handy if we type hint the CollectorRegistry, allowing Laravel to automagically create the class for us.

I also like to create a Facade for this use case, as Facades give you easy ways to swap out to a fake when running tests. This is great if your tests run a code path that triggers a metric creation.

To do that, I create a file app/Prometheus/Prom.php with the following:

    <?php

namespace App\Prometheus;


use Prometheus\CollectorRegistry;
use Illuminate\Support\Facades\Facade;

class Prom extends Facade
{
    public static function fake()
    {
        static::swap(new Fake);
    }

    protected static function getFacadeAccessor()
    {
        return CollectorRegistry::class;
    }
}

  

Now we can call Prom::whatever() and have that pass off the method call to the CollectorRegistry. In tests, we can call Prom::fake() so that a fake CollectorRegistry is called that doesn’t do anything.

The Fake class has __call and __callStatic magic methods that do nothing.

Metrics Endpoint

Let’s next see how to create a /metrics endpoint. This part is pretty easy - there’s a convenient few function calls we make to have the library generate the output for the metrics.

If we register route to an invokable controller via Route::get('/metrics', MetricsController::class), the controller just needs something like the following:

    <?php

# generated via 
# php artisan make:controller -i MetricsController
namespace App\Http\Controllers;

use App\Prometheus\Prom;
use Illuminate\Http\Request;
use Prometheus\RenderTextFormat;

class MetricsController extends Controller
{
    public function __invoke(Request $request)
    {
        $formatter = new RenderTextFormat;

        return response(
            $formatter->render(Prom::getMetricFamilySamples()),
            200,
            [
                'Content-Type' => RenderTextFormat::MIME_TYPE,
            ]
        );
    }
}

  

You may want to add a middleware to this so the route is not accessible to the entire world.

Collecting Metrics

Collecting metrics is fairly easy - we can generate a counter (or gauge, or whatever), and then call a method on it such as inc() or set() to set a value.

The interesting part is labels! You can define what labels exist when generating the metric (counter, gauge, etc). Then you must set those labels values when adding the metric.

That looks like this:

    // We get/create a counter: $namespace, $metric_name, $description, $labels
$counter = Prom::getOrRegisterCounter('chipper', 'build_count', 'Number of CI builds', ['team', 'platform']);

// Increment counter "build_count" by 1, setting values for labels "team" and "platform"
$counter->inc([$team->getKey(), 'fly']);

We can create (or get) a counter metric. This requires a few parameters, including setting the label..uh..labels - the “keys” (rather than the values). I set “team”, which is a numerical team ID, and “platform” because Chipper CI has 2 build platforms (an old one based on a Nomad cluster, and a newer one based on Fly Machines).

Each time we increment this counter, we need to set those label values. We see that in the 2nd line there, where we pass the Team’s ID, and the platform used.

The gauge metric is very similar:

    // We get/create a gauge: $namespace, $metric_name, $description, $label(s)
$gauge = Prom::getOrRegisterGauge('chipper', 'billable_builds', 'Number of CI billable builds', ['team']);

// Increment guague "billable_builds" by 1, setting value for label "team"
$gauge->set(42, [$team->getKey()]);

Just like counters, gauges can inc() or incBy(). They also have a method set() to set an exact value.

Once you know what you want to measure in your application, you can call those methods as needed.

Fly.io ❤️ Laravel

Try out Fly, get free metrics. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Fly Configuration

Once your application has metrics to collect, we need to view those. Rather than setup Prometheus and Grafana yourself, Fly.io will do this for you.

You can add something like the following to your fly.toml:

    [metrics]
  port = 8080
  path = "/metrics"

  

Fly.io will then scrape that endpoint. Be sure to get the port right - by default, Laravel apps run on port 8080 inside the VM (port 80 and 443 are exposed to the outside world, but Fly.io uses the internal private network).

If your application does stuff like redirect all requests from “http” to “https://”, you may run into issues.

Within a few minutes after a successful deployment, you should be able to spot the /metrics endpoint getting hit occasionally in your access logs.

Grafana

When that’s setup, and metrics are getting collected, you can start to make some graphs within https://fly-metrics.net!

You can create a new dashboard, and add a graph (click the “add a new panel” button) to the dashboard.

When creating a graph, what I do is two-fold:

Create a variable (it’s a Dashboard setting)
Create a graph that can use that variable

The Variable

Here I created a variable named “team”, which will let me filter by the team label.

The important parts here: We get all unique values for label team via label_values(team). Then we include an all option which is a regex wildcard card .*.

A Graph

Then I can create a graph that makes use of that variable:

Setting the data source as Prometheus on Fly will auto-populate the Metric dropdown. You can select a custom metric, and then start playing with aggregation functions and so on. Here we graph the rate of increase, which will help spot potential issues of large spikes in build counts.

Note that we use =~, which sets the team label as a regex. That allows the “all” option of .* to work so any/all teams are included when all is used as a dashboard filter. The variable also lets you filter by specific label!

Invoice PDF generation with Browsershot

2023-04-20T00:00:00+00:00

Invoicing is boring and manually writing up invoices even more so, that’s why you should automate the entire process in your app. And why not run it on our servers while you’re at it? With Fly.io, you can get your Laravel app running globally in minutes!

Invoicing is a hugely important part of any business. By having your Laravel app generate invoices automatically, you can make the life of your users a lot easier and we all know that means they’ll hang around longer and be more keen to use your apps.

In this article, I’ll show you how to use Spatie’s Browsershot package to generate invoice PDFs automatically. Read on to find out how!

Making the invoice view

Before we talk about generating PDF’s or how Browsershot is set up, we’ll need something to show on our invoice. So I set up a fictional company called The Invoicing Company™️ with the sole purpose of sending out invoices, a solid business plan if you ask me. Anyway, here’s how the invoice looks:

To generate the pdf, we’ll need a web page that only displays the pdf itself. So, I set up a view that shows an A4-sized div like this: <div style="width: 210mm; height: 297mm; padding: 25mm 19mm"></div> . Once that’s set up, you can let your inner web designer run free.

Fly.io ❤️ Laravel

Fly your servers close to your users—and keep your apps running buttery smooth. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Setting up Browsershot

For turning HTML into a PDF, I’m using Spatie’s Browsershot package. It has great documentation which made it a joy to use. I had to make some changes to the Dockerfile that’s shipped with the fly launch command as well, this is only needed for the worker and not for the web app itself:

    # Other Dockerfile stuff here...

# Packages like Laravel Nova may have added assets to the public directory
# or maybe some custom assets were added manually! Either way, we merge
# in the assets we generated above rather than overwrite them
COPY --from=node_modules_go_brrr /app/public /var/www/html/public-npm
RUN rsync -ar /var/www/html/public-npm/ /var/www/html/public/ \
    && rm -rf /var/www/html/public-npm \
    && chown -R www-data:www-data /var/www/html/public

+ # ADDED: Browsershot does need npm and nodejs. So we'll install them anyway in the base container.
+     # I also added the chromium dependencies, since they don't get installed correctly.
+ RUN curl -sLS https://deb.nodesource.com/setup_$NODE_VERSION.x | bash - \
+     && apt-get install -y nodejs gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgbm1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget libgbm-dev libxshmfence-dev \
+     && npm install -g npm \
+     && npm install --unsafe-perm puppeteer

EXPOSE 8080

ENTRYPOINT ["/entrypoint"]


  

Making the job

Since it’ll take some time and we don’t want to make our users wait around, we’ll be using queued jobs to generate the pdf. Here’s how I set it up: I made a Job that generates the pdf, and then sends a post request to a signed url to upload the invoice to the correct Order. I’m keeping it simple, you can use S3 here or something similar you like. The reason I added this post request is because our worker will be running in a different VM than our web app, so it can’t reach the web app’s file system. So here’s how the job’s handle method looks:

    // Add a private property $order here and set it in the __construct() method.

/**
 * Execute the job.
 *
 * @return void
 */
public function handle()
{
    try {
        Browsershot::html(view('invoice.invoice', ['order' => $this->order])->render())
            ->noSandbox()
            ->waitUntilNetworkIdle()
            ->format('A4')
            ->showBackground()
            ->savePdf("temp.pdf");

        $postRoute = URL::signedRoute('orderinvoices.store', ['order' => $this->order]);
        Http::attach('invoice', file_get_contents('temp.pdf'), 'invoice.pdf')
            ->post($postRoute)
            ->throw();
    }
    catch (\Exception $exception )
    {
        Log::error($exception);
    }
}

  

Some things to notice here: The noSandbox() method is needed to avoid errors with the headless Chrome browser used by Puppeteer. Because I created the HTML code we pass along myself, I’m not too worried about the security risks. You can read more about Chrome sandboxing here.

What’s next?

By now, we’ve covered everything needed to create a job that generates PDFs with anything you want on them. So what do we do now? My humble suggestion is that you check out my article on cost-effective queue workers. I explain how to set ‘em up and more importantly, how running your queue workers on Fly.io can save you money. See you there!

Sharing Google Maps Data across Separate Livewire Components

2023-04-20T00:00:00+00:00

Deploy now on Fly.io, and get your Laravel app running in a jiffy!

Let’s say we have separate Livewire components for:

A Google Map Element - an interactive map for adding and deleting location markers
A Search box Element - to re-focus the Google Map element to a user given location
A Drop Down Element - to filter markers shown in the Google Map element
Finally, a Form - to submit user provided data, from all the components above

How exactly, do we share data between these separate Livewire components?

In this article, we’ll chance a glimpse on different possibilities for sharing data across Livewire components:

Sharing Data through Livewire's dispatchBrowserEvent: useful when data from a component A in the server is needed to make changes to a component B’s UI
Sharing Data through JavaScript Variables: useful when data gathered from component A’s UI is needed to make changes to a component B’s UI
Sharing Data through Livewire's emit events: useful when processed data from a component A in the server is needed in a component B’s component in the server
Sharing Data through Html Query Selectors: useful when we need to submit data from user input provided in separate components’ elements in parent component

We’ll go through all these different possibilities by integrating these Livewire components together: a Google Maps Element, a Search Box, a Drop Down Filter, and a Form.

Creating the Google Map Component

Let’s start with the center stage of our article today: A Google Map Component. We’ll use this to allow our users to add markers to different locations in the world.

First, create the component with php artisan make:livewire map. Make sure to add in its view a div element with a specified width and height to show the map:

    <!--app\resources\views\livewire\map.blade.php-->
<div>
    <div 
    wire:ignore id="map" 
    style="width:500px;height:400px;">
    </div>

  

Then add the script to initialize a Google Maps element into this div. Make sure to authorize access to Google Maps api by either using the inline bootstrap loader or the legacy script loading tag.

        <script>
        /* Add Inline Google Auth Boostrapper here */

        /* How to initialize the map */
        let map;
        async function initMap() {
            const { Map } = await google.maps.importLibrary("maps");
            map = new Map(document.getElementById("map"), {
                    zoom: 4,
                    center: { lat: @js( $lat ), lng: @js( $lng ) },
                    mapId: "DEMO_MAP_ID",
            });
        }

        /* Initialize map when Livewire has loaded */
        document.addEventListener('livewire:load', function () { 
            initMap();
        });
    </script>
</div>

  

Notice the @js( $lat ) and @js( $lng ) in the code snippet above? That’s Livewire’s helper that let’s us use PHP attributes $lat and $lng in our view’s JavaScript. We’ll have to declare those from the Map component:

    /* App\Http\Livewire\Map.php */

class Map extends Component{

      public $lat = -25.344;
      public $lng = 131.031;

Above, we have declared default location coordinates. This would show a default location in our maps component. In the next section below, we’ll add in a search box component to allow users to easily relocate focus to their desired location. And from there, we’ll implement the first way to share data across components.

Create a separate component with: php artisan make:livewire map-search-box. It’s view will have two elements, an input text field and a button:

    <!--app\resources\views\livewire\map-search-box.blade.php-->
<div>
    <input type="text" wire:model.defer="address" />
    <button wire:click="search">Search</button>
</div>

  

The text element is wired to an $address attribute of the component, and uses model.defer to make sure Livewire doesn’t send its default requests whenever a change occurs on the element. On the other hand, the button element is wired to a search() method in the Livewire component through a click listener.

This search() method converts the string value of the input element( wired to $address ) into location coordinates:

    /* App\Http\Livewire\MapSearchBox.php */

class MapSearchBox extends Component{

      public $address;

      public function search()
      {
          // Use a custom service to get address' lat-long coordinates
          // Either through Google GeoCoder or some other translator
          $coordinates = new \App\Http\Services\GoogleLocationEncoder( 
            $this->address 
          );
      }

  

After retrieving our coordinates from the MapSearchBox component, we’ll have to re-center the location visible from our Map component’s view.

Hmmmm. But. The MapSearchBox component is a separate component from the Map component, so—how exactly do we share data between two separate components?

A usual way to share event data between components is through Livewire’s emit feature. However, this approach actually makes two immediate requests for every emit: the first request a call to the component the event was emitted from, and the second request a call to the component listening for the emitted event.

In our use case( recentering the map based on the coordinates from the search box ), we don’t actually need the second request. We only need the resulting coordinates to be shared to our Map‘s UI to change the location it’s map is centered on.

So, instead of emit, let’s use Livewire’s dispatchBrowserEvent:

    /* \App\Http\Livewire\MapSearchBox */
public function search()
{
    /* Get coordinates */

    // Dispatch event to the page
+  $this->dispatchBrowserEvent( 'updatedMapLocation',[
+    'lat' => $coordinates->getLatitude(),
+    'lng' => $coordinates->getLongitude()
+  ]);
}

  

The browser window event, updatedMapLocation from the MapSearchBox, is fired to the current page, where all available components can listen in on. Since our Map component’s view is also in this page, it can easily listen to the dispatched event and re-center the location based on the received coordinates:

    /* \app\resources\views\livewire\map.blade.php */
<script>
// Listen to location update from search box
window.addEventListener('updatedMapLocation',function(e){
    // Defer set lat long values of component
    @this.set( 'lat', e.detail.lat, true);
    @this.set( 'lng', e.detail.lng, true);

    // Translate to Google coord
    let coord = new google.maps.LatLng(e.detail.lat, e.detail.lng);

    // Re-center map
    map.setCenter( coord );
});

  

This takes just one request to the MapSearchBox component to process coordinates from the user given string and update the Map component’s view to re-center its location—neat!

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Revising the Maps Component for Pin Marking

An important functionality of map elements is their “Pin Dropping” feature, allowing users to mark locations in the map. For this, we’ll need a click event listener in our map element:

    /* \app\resources\views\livewire\map.blade.php */
let map;

// Dictionary of markers, each marker identified by its lat lng string
+  let markers = {}; 

async function initMap() {
  /* Map initialization logic here... */

  // Add marker listener
+  map.addListener("click", (mapsMouseEvent) => {
      // Get coordinates 
      let coord = mapsMouseEvent.latLng.toJSON();

      // Generate id based on lat lng to record marker
      let id = coord.lat.toString()+coord.lng.toString();

      // Add Marker to coordinate clicked on, identified by id
      markers[id] = new google.maps.Marker({
          position: mapsMouseEvent.latLng,
          map,
          title: "Re-Click to Delete",
      });

      // Delete marker on re-click
      markers[id].addListener("click", () => {
          markers[id].setMap(null);
          delete markers.id;
      });
  });
});  

  

Step by step, what’s happening above? First, we add a JavaScript dictionary called markers to hold reference to uniquely identified location markers in the Map component’s view. Then, we revise the initMap() functionality to intercept click events on the map.

For each location clicked on, we get the coordinates, generate a unique id based on these coordinates, and assign this unique id as a key in our markers dictionary, setting its value as a reference to a new Google Map marker. We also add an on-click listener to each new marker to optionally delete them on re-clicking.

In the next section, we’ll move on to our second method of sharing data, by filtering the markers above through a separate filter component.

Creating the Filter Component

Let’s say we want a way to bulk remove map markers in the Map component based on filter conditions from the server. We can add a component for this: php artisan make:livewire map-marker-filter.

In this example, we’ll remove markers that are not within the confines of an “area”. We’ll use these “area” options to filter the map markers. To get these options, we can declare them from the MapMarkerFilter component:

    /* \App\Http\Livewire\MapMarkerFilter */

class MapMarkerFilter extends Component
{
    public $options = [
        ['id'=>1, 'label'=>'Area1'],
        ['id'=>2, 'label'=>'Area2'],
    ];

  

We can provide these “area filters” as $options in a select element in the view:

    /* app\resources\views\livewire\map-marker-filter.blade.php */
<div>
  <select id="area" onchange="filterChange( this )">
    <option>Select an Area</option>
    @foreach( $options as $opt )
        <option value="{{ $opt['id'] }}">
        {{ $opt['label'] }}
        </option>
    @endforeach
  </select>
</div>

  

When a new filter is selected, we’ll send two things to the MapMarkerFilter component in the server: 1️⃣the value of the selected option, and, 2️⃣the coordinate list from the JavaScript markers dictionary.

We can easily get the 1️⃣selected option on change, since it’s in our current MapMarkerFilter component. But how about the JS variable 2️⃣markers holding our pins that’s from the Map’s view? It’s declared in a different component, so how do we share this to the current MapMarkerFilter’s view?

To share the markers list from Map to MapMarkerFilter, let’s try a straightforward approach; let’s look into the scope the markers variable is available in. Being declared in the JavaScript of the Map component’s view, let’s see if we can get this value from the JavaScript of MapMarkerFilter.

    /* app\resources\views\livewire\map-marker-filter.blade.php */
<script>
  function filterChange( objVal ){
      let filterId = objVal.value;
  +   let coords = Object.keys(markers);
  +   console.log( coords );
  +   @this.filterMarkers( filterId, coords );
  }
</script>

  

Save the changes, and select an option from the filter…and. And—it works! We actually have access to Map’s JS variable marker from MapMarkerFilter! No shocker here. These are JavaScript variables scoped and available to other scripts within the same page:

Notice the @this.filterMarkers() statement. It’s an inline way for calling a Component method from the view’s JavaScript. In the case above, two variables, filterId and coords from the JavaScript view is sent to a filterMarkers() method in the component:

    /* \App\Http\Livewire\MapMarkerFilter.php */
public function filterMarkers( $filterId, $coords )
{
    // Using filterId, get marker ids that should be removed from the map
    // $toRemove sample: ["-19.19356730928235_125.40645731663705", "..."]
    $toRemove = (new \App\Http\Services\MapMarkerFilter)
    ->getCoordsToRemove( $filterId, $coords );

    // Send this back to the view
    $this->dispatchBrowserEvent( 'removeMarkers', [
        'coords' => $toRemove
    ]);
}   

  

From this method, we can use the selected $filterId to determine which markers in the $coordinates list is to be removed from the map view.

Once we’ve determined the marker coordinates to remove, we can use another dispatchBrowserEvent call to fire a removeMarkers browser event back to the client page. Our markers are in the Map component, and so, from its view’s JavaScript, let’s add a listener to this event, and remove the specified markers id sent from the event:

    /* app\resources\views\livewire\map.blade.php */

/* Listen to location update from search box */
window.addEventListener('removeMarkers',function(e){
    // Delete each coordinate by id
    for( i in e.detail.coords ){
        let id = e.detail.coords[i];
        markers[id].setMap(null);
        delete markers[id];
    }
});

  

The Form Finale

For the finale, let’s submit all user input, from each components we’ve declared above: 1️⃣the search keyword from the Search component, 2️⃣the filter option selected from the Filter component, and finally, 3️⃣the map markers selected in the Map component.

Let’s create a form component with php artisan make:livewire map-form. For its view, we simply include all other components we’ve created, with an additional button:

    <!--app\resources\views\livewire\map-form.blade.php-->
<div> 
  <h1>Form</form>
  <form wire:submit.prevent="submit">
      <div class="flex flex-row justify-between">
          <livewire:map-search-box />
          <livewire:map-territory-filter />
      </div>

      <livewire:map />
      <button type="submit" class="btn btn-primary">Submit</button>
  </form>
</div>

  

Now, we can certainly do an emit submit signal from parent that all child components will listen to, and emit up their values in response to:

    /* App\Http\Livewire\MapForm.php */
public function submit(): void
{
    $this->emit('submit');
}

/* App\Http\Livewire\<somechildcomponenthere> */
// Listen to submit from parent
public $listeners = [
    'submit' 
];

public function submit(): void
{
    $this->emitUp('map-form', $this->valueNeedToPass);
}

  

But. Thats gonna be a ton of request, one request from the form, and two requests to-and-fro each listening component (1. to receive the parent emit event, and 2. to send value to parent component ). Ah-yikes.

A'ight, since the above ain’t so thrifty, let’s try a different approach on sharing, shall we?

Similar to how js variables are available from one component to another component in the same page, so are html elements! So, we simply get the values of the input elements from the MapSearch and MapMarkerFilter components in our MapForm’s JavaScript.

First, revise the form element to call a JS function:

    <!--app\resources\views\livewire\map-form.blade.php-->

- <form wire:submit.prevent="submit">
+ <form onsubmit="return process()">

Then, make sure to add an identifier to their input elements:

    <!--app\resources\views\livewire\map-search-box.blade.php-->
<input id="searchString" type="text" placeholder="Location" wire:model.defer="address" />

<!--app\resources\views\livewire\map-marker-filter.blade.php-->
<select id="filterId" onchange="filterChange( this )">

Since markers is available as a variable in the current page, we can simply call it from MapForm’s JavaScript as well:

    function process()
{
  // Get Search keyword
  @this.set('searchKey', document.getElementById('searchId').value, true);

  // Get Selected Filter option
  @this.set('filterId', document.getElementById('filterId').value, true);

  // Get markers keys list, as the key themselves are the coordinates
  @this.set('coords', Object.keys(markerList), true);

  // Trigger Submit
  @this.submit();

  return false;
}

  

Then simply define the submit method in our MapForm component:

    /* \App\Http\Livewire\MapForm.php*/

public $searchKey;
public $filterId;
public $coords;

public function submit()
{
    // Validate entries
    $this->validate([
      'searchKey' => 'string',
      'filterId'  => 'numeric',
      'coords'    => 'required',
    ]);

    // Do processing

    // Redirect 
}

  

Before calling the submit() method in the server, we first set the $searchKey,$filterId, and $coordsvalues from different components through query selection, and JavaScript variables. And now, we all have these values from separate components in our parent MapForm component!

Learning Possibilities

So, what did we learn today?

—Possibilities!

We learned about four, different, use-case-helpful possibilities in sharing data amongst Livewire components.

And although, they’re not all pure Livewire specific approaches, they do the job pretty nicely in their specific use case domains. Try them out sometime!

Cost-Effective Queue Workers With Fly.io Machines

2023-04-14T00:00:00+00:00

We’re going to save you money today by running a queue worker in a Fly.io machine. Read on to see how exactly this saves money and how I set it up. With Fly.io, you can get your Laravel app running globally in minutes!

Fly.io machines have been announced and have been made the default way new apps will be run on Fly.io’s platform. They have many advantages, and one of them is their super-fast boot times.

In this article, I’ll show you how to use machines to run both the web app and a queue worker in the same app. I’ll also save you money in the process, so let’s dive in!

Setting up the web app and worker

First things first: I will be talking about apps and machines here.

By apps I mean applications on Fly.io, they are created by running fly launch and they have a fly.toml file that holds their configuration. An app contains one or more machines.

Machines are VMs that belong to a specific app, and they have some special Fly.io sauce on top: The machines API. This is a REST api that allows you to manipulate these machines in all kinds of ways: update, scale, destroy, restart, clone,… You name it!

The cool thing about machines in an app is that they can run different commands. So I can create one app that contains two machines, one for the web app and one for the queue worker. This is set up in the fly.toml using process groups. The app process group is the default, and will be set up already if you ran fly launch. I added a worker process group by adding it to the processes tag:

    [processes]
app = ""
worker = "php artisan queue:work --stop-when-empty"

  

The app will run whatever is in the Docker entrypoint, and the worker will run the php artisan queue:work --stop-when-empty command. This will keep the worker running as long as it’s fed with new jobs to process. When it’s idle, the machine will stop gracefully and stop slowly draining your wallet. Good stuff!

How do we get the machine running again though? That’s where the machines API comes in: If we know the ID of the worker machine, we can just run a CURL command to start the machine after we added in a job. That’s nice, but it could be even more convenient! I created an artisan command to start a machine, and I’ll run it whenever I dispatch a job.

If you need an example of a job that would be great fit to test our shiny new machines, check out my article on invoice PDF generation with Spatie’s Browsershot package!

Starting a machine with an artisan command

Here’s the machines API endpoint I’ll use to start a machine:

curl -i -X POST \
-H "Authorization: Bearer ${FLY_API_TOKEN}" -H "Content-Type: application/json" \
"http://${FLY_API_HOSTNAME}/v1/apps/${FLY_APP_NAME}/machines/${FLY_MACHINE_ID}/start"

There’s 4 variables that need to be filled in:

FLY_API_TOKEN: the api token to authenticate on fly.io. Run fly auth token to get it.
FLY_API_HOSTNAME: the hostname of Fly.io’s machines API. _api.internal:4280 when connected to Fly.io’s internal network (in apps running on Fly.io for example) or 127.0.0.1:4280 if you’re using flyctl proxy.
FLY_APP_NAME: The name of your app on Fly.io.
FLY_MACHINE_ID: The ID of the worker machine.

I’d suggest putting the last three in the [env] section of fly.toml, since they will never change much. The FLY_API_TOKEN won’t change as well, but that’s quite sensitive so I’d use secrets for that.

I wanted my artisan command to be generic, so I set the machine ID as input on the command. It’ll be pulled from the env variables anyway, but this leaves room for multiple workers that handle different queues and/or connections. Here’s how it looks:

        protected $signature = 'machine:start {id : the ID of the machine to be started.}';

    protected $description = "This command starts a Fly.io machine. It needs the machine's ID as input.";

    /**
     * Execute the console command.
     *
     * @return int
     */
    public function handle()
    {
        $flyApiHostname = "http://_api.internal:4280";
        $flyAuthToken = env("FLY_AUTH_TOKEN");
        $flyAppName = env("FLY_APP_NAME");

        $machineId = $this->argument('id');
        $response = \Http::withHeaders([
            'Authorization' => "Bearer $flyAuthToken",
            'Content-Type' => 'application/json'
        ])->post("$flyApiHostname/v1/apps/$flyAppName/machines/$machineId/start");

        if ($response->failed())
        {
            $this->error($response);
            return Command::FAILURE;
        }

        $this->info($response);
        return Command::SUCCESS;
    }

  

Fly.io ❤️ Laravel

Run your queue workers on Fly.io and use the machines API to start, stop, scale, update and destroy them. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Running the artisan command whenever a job is dispatched

If I can get that machine:start <ID> command to run every time this job is dispatched, then the worker machine will boot up every time a job is added to the queue. Great stuff! I did this by extending the dispatch() method on the Dispatchable trait that my job uses.

  // in the Job class
-  use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
+  use Dispatchable {dispatch as traitDispatch;}
+  use InteractsWithQueue, Queueable, SerializesModels;

  // other stuff here...

  + public static function dispatch(...$arguments)
  +     {
  +         Artisan::call('machine:start', ['id' => env('FLY_WORKER_ID')]);
  +         self::traitDispatch(...$arguments);
  +     }

Notice that I’m not overriding the dispatch() method, I’m extending it: I do what I need to do and then I call self::traitDispatch(...$arguments) which runs the original dispatch() method. This way, the Artisan call: command is executed every time right before the job is actually dispatched. Nice!

That’s a wrap!

By now, I’ve talked about Fly.io’s machines a lot: What they are, what makes them special and how to use the machines API. Since I want to make things as easy as possible, I’ve also laid out how to start a machine whenever needed using an Artisan command.

This all comes together to set up a machine that only runs when it needs to. This saves on computing resources which is good for the environment but more importantly: it saves you money. Now go pitch using Fly.io machines to your boss and get that raise you know you deserve!

Tricks for Running Commands with Laravel Process

2023-04-13T00:00:00+00:00

Fly.io is a great place to run both apps and workers! Deploy now on Fly.io, and get your Laravel app running globally in a few commands!

Laravel 10 has released the Laravel’s Process facade, which make running external commands super easily.

    $result = Process::run("php -v");
echo $result->output();

This uses Symfony Process under the hood, and adds a TON of quality-of-life improvements.

What might you run commands for? I’ve personally used it to run docker commands on an early version of Chipper CI, and recently have seen it used to run ffmpeg commands to edit media files. There’s a lot of times when it makes sense to reach out beyond PHP!

Let’s see how to run commands (processes), and learn some tricks along the way.

The Process Component

Some basics first! Let’s run the ls -lah command to list out files in the current directory:

    $result = Process::run('ls -lah')

  

The $result is an instance of the Illuminate\Contracts\Process\ProcessResult interface.

If you check that out, we’ll see a few handy methods available to us:

    # See if it was successful
$result->successful(); // zero exit code
$result->failed(); // non-zero exit code

$result->exitCode(); // the actual exit code

$result->output(); // stdout
$result->errorOutput(); // stderr

  

Stdout and stderr

Output from a command will come in 2 flavors: stderr and stdout. They’re sort of like different channels that a command can use to report information, errors, or important data.

Did you know that stderr isn’t just error messages? There’s some important stuff to know!

In general, stderr output will be human-readable output meant to inform users about errors or just general information.

The stdout output is generally meant to be something either human-readable or machine readbable. It might be something you’d ask some code to parse.

Let’s see that in action.

If we run fly apps list -j to get a JSON format list of apps and their information, we may see this:

The informational message about the version of flyctl and how to update is output to stderr. The real content (the list of apps) is sent to stdout.

The stdout output is what we care about. Luckily, if I wanted to pipe the JSON output to jq for some extra parsing, I could! Even though the command outputs an informational message, sending output through a pipe will only pass along the stdout content.

    # Find the status of each app
# The pipe `|` only get stdout content
fly apps list -j | jq '.[].Status'

  

That lists the status of each Fly.io app listed.

Parsing Output in PHP

We can use this knowledge to parse command output in PHP.

    // Get our list of apps as JSON and parse it in PHP
$result = Process::run("fly apps list -j");

if ($result->successful()) {
    $apps = json_decode($result->output(), true);

    var_dump($apps);
}

  

Here’s a fun issue with the above code: The flyctl output has some debug information sent through stdout, so we can’t directly parse the JSON outout!

It turns out that when running the above in a tinker session, the Laravel environment has LOG_LEVEL=debug set. While that is a Laravel-specific environment variable coming from the .env file, flyctl happens to use it also!

Why is the fly command seeing that? This brings us to our next topic!

Environment Variables

Processes typically inherit their parent-processes environment variables. Since our PHP tinker session has environment variable LOG_LEVEL set, that’s getting passed to the fly command we ran. The fly command just so happens to use that environment variable, and so we get some debug information output (to stdout in this case).

Luckily, we can unset that environment variable by setting it to false:

    # Unset the LOG_LEVEL env var when 
# running the process:
$result = Process::env(['LOG_LEVEL' => false])
    ->run("fly apps list -j");

if ($result->successful()) {
    $apps = json_decode($result, true);

    // todo: Ensure JSON was parsed
    foreach($apps as $app) {
        echo $app['Name']."\n";
    }
}

  

Now our output doesn’t contain the debug statements in stdout and we can parse the JSON in PHP!

We’re not limited to unsetting environment variables. We can pass additional ones (or overwrite others) by passing more things to that array:

    # If you want to see a TON of debug
# output from the `fly` command:
$result = Process::env([
        'LOG_LEVEL' => 'debug',
        'DEV' => '1',
    ])->run("fly apps list -j");

  

Fly.io ❤️ Laravel

Fly your servers close to your users—and keep your apps speedy. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Streaming Output

What if we run a command that does a bunch of stuff over time, like deploying a Fly app?

We can actually get a command’s output in “real time”. We just pass the run method a closure:

    $result = Process::env('LOG_LEVEL' => false,)
    ->path("/path/to/dir/containing/a/fly.toml-file")
    ->run("fly deploy", function(string $type, string $output) {
        // We'll ignore stderr for now
        if ($type == 'stdout') {
            doSomethingWithThisChunkOfOutput($output);
        }
     });

  

I’ve used this to stream output to users watching in the browser (with the help of Pusher and websockets).

Security

I’m not going to talk about sanitizing user input and the dangers of allowing user input to define what commands you might run (yeah, probably don’t do that).

Instead, I have something specific to Fly.io and Laravel! If you ran fly launch to generate a Dockerfile, we have something to tweak here.

By default, the open_basedir setting is set in the php-fpm configuration. This limits what directories PHP can see, which prohibits the ability for PHP to see (and run) commands in /usr/local/bin or similar.

You change that, you can either remove that configuration or append a directory containing a command you’ll be using.

In file .fly/fpm/poold/www.conf, I can append directory /usr/local/bin:

    ; @fly settings:
; Security measures
php_admin_value[open_basedir] = /var/www/html:/dev/stdout:/tmp:/usr/local/bin
php_admin_flag[session.cookie_secure] = true

  

That change should get pulled in in your next deployment.

There’s More

There’s a bunch more you can do! Check out the docs and examples here!

No-Upload, Batched Import of Data with SheetJS CE and Livewire

2023-04-06T00:00:00+00:00

Deploy now on Fly.io, and get your Laravel app running in a jiffy!

When we think about first steps in importing spreadsheet data, we usually think of uploading the file first to the server, before processing the import.

But. What if we didn’t need to upload the file at all?

In this article, we’ll import data from our spreadsheet file, but completely skip file upload to the server. Instead, we’ll directly send batches of data rows from our selected spreadsheet file in the browser to our server—easily and quickly, with the help of SheetJS CE, Web Workers, and Livewire!

Version Notice. This article focuses on Livewire v2, details for Livewire v3 would be different though!

The Setup

Here’s a Github Readme pointing to relevant files we’ll be using in our setup today.

To start off, create a Livewire component by running: php artisan make:livewire excel-importer. Our view should contain an input element users can upload spreadsheet files to, and a button they can click on to submit their file for processing:

    <div>
    <h1>Import Excel Data</h1>
    <form onsubmit="return process()">
        <input type="file" id="myFile" />
        <input type="submit" />
    </form>
</div>

  

Next, let’s add in SheetJS CE to our view. There’re various ways we can include SheetJS CE into our project, the quickest being the use of its “standalone browser script”:

    <script lang="javascript" src="https://cdn.sheetjs.com/xlsx-latest/package/dist/xlsx.full.min.js" ></script>

  

Once the user selects a spreadsheet file, and clicks on the submit button, we’ll do the following:

Extract row data from the file using SheetJS CE
Send row data to the server using Livewire
Validate row columns with Livewire’s validate method
Queue a job to process the validated rows in the background

Along the way we’ll also make changes to possibly accommodate large file processing:

Use of web workers to read and parse file
Batched sharing of rows with Livewire
Use of another library Papa Parse for csv files

Extracting and Parsing Data

When a user submits their file for processing, the process() function will be called. This reads the file’s content as an array of bytes, called “ArrayBuffer” using the FileReader api:

    /* resources\views\livewire\excel-importer.blade.php */
<script>
  // Reader and File Reference
  var reader = new FileReader();
  var fileElement = document.querySelector('#myFile');

  function process(){
    // Content as an ArrayBuffer
    reader.readAsArrayBuffer(fileElement.files[0]);

  

Next, we’ll have to make sense of these “raw” bytes from our spreadsheet file, transform it into readable, row data. But how do we do that?

There are all kinds of spreadsheet types out there—CSVs, XLSXs, XLSs, and so forth—and they have different ways for parsing their content. Luckily for us, SheetJS CE accommodates parsing a multitude of spreadsheet file formats, and so, it’s one of our superstars for today. We’ll use it to parse spreadsheet content into an array of rows.

Specifically, we’ll use its magic parser, XLSX.read, to parse its raw content:

        reader.onload = function(event) {
        // Parse raw content
        var workbook = XLSX.read( event.target.result );
        // Release after read
        event.target.result = null; 

  

Spreadsheets can contain more than one sheet, and SheetJS CE handles this by storing sheet names in its SheetNames array. We can use each name as a key to access their corresponding sheet content. Then, with our sheet content available, we can format this to an array of arrays using XLSX.utils.sheet_to_json:

            workbook.SheetNames.forEach(function(sheetName) {
            // Current sheet
            var sheet = workbook.Sheets[sheetName];

            // Include header param to return array of arrays
            var rows  = XLSX.utils.sheet_to_json(sheet, {header:1});
        });
    } // End reader.onload 
  } // End process()
</script>

  

Passing {header:1} above to our call to sheet_to_json() allows us to receive the rows as an array of arrays. Of course, there are other various output formats, you can read more starting from this section. Here’s a sample output of our rows, as an array of arrays:

Now that we’ve extracted our data, it’s time to start our import with Livewire.

Livewire offers a seamless bridge to share JavaScript-friendly data, and method calls between client and server. As a result, we don’t need to create a separate route and controller just for importing our data. We can simply call a method in the server component from client JavaScript, and pass certain data types, like so:

          // Include header param to return array of arrays
      var rows  = XLSX.utils.sheet_to_json(sheet, {header:1});
      rows.shift();// Remove header
+     @this.importData(rows)
+     rows = null; // Release 

  

The above should make a call to the server triggering a method named importData, and pass the rows variable as a parameter. Here’s how the method will look in our component:

    /* \app\Http\Livewire\ExcelImporter  */
public function importData( $rows ){}

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Now that we have a hold of our imported rows in the server, we’ll have to make sure that they’re data we can process—it’s time for some validation.

Validating An Array of Arrays

Before we can process the rows received in our server, we’ll have to validate its column values first. Livewire reserves the $rules attribute in any Livewire controller, which we can use to add validation rules. We can validate each row in a generic manner like so: rows.*.

Now, let’s say, each 0th column within a row should always be available, and at the same time should be of some numeric value like:[[23],[12]]. To validate this rule, simply indicate the rule for the 0th index’s rule:

    /* \app\Http\Livewire\ExcelImporter  */
protected $rules = [
  'rows.*.0' => 'required|numeric'
];

  

Once we’ve set up validation, let’s proceed with the pass or fail part. Livewire offers the validate() method to do a pass or fail on its public attributes’ rules listed in $rules.

In our case, our rule is intended for a rows attribute, and so we declare that as a public attribute, and set its value. Once it’s set with the imported data, we trigger validation on it:

    +   public $rows;

public function importData( $rows ){
+    // Set rows attribute first
+    $this->rows = $rows;

+    // Now pass or fail
+    $this->validate();

+    // Don't send back rows in response
+    $this->rows = [];
}

  

We can catch any validation errors in our view like so:

    @foreach( $errors->all() as $error )
  <div> {{ $error }} </div>
@endforeach

  

And in case we want to record the errors in the server, simply use a try catch block:

    /* \app\Http\Livewire\ExcelImporter  */
try{
  //$this->validate()
}catch(\Illuminate\Validation\ValidationException $e){
  $this->rows = []; // Don't send back rows in response
  foreach( $e->validator->errors()->all() as $error  ) 
    Log::info( 'found error'.$error );
  throw $e;
}

  

Queued Processing of Imported Data

Processing rows of data can take some time to complete, and would send our users into a spiraling dread of pause before they can move forward from their import duty.

Instead of making our users wait for this prolonged pause, we’ll send data we received to a job, and queue this job for processing in the background. This allows us to promptly respond with a processing status, without waiting for the process to complete.

Create a job with php artisan make:job ImportExcelDataJob. This should generate our Job file at app\Jobs\ImportExceltDataJob. We can call this from our Livewire component like so:

Make sure to import App\Jobs\ImportExcelDataJob in our component!

    /* app\Http\Livewire\ExcelImporter  */
public function importData( $rows ){
     $this->rows = $rows;
     $this->validate();
     $this->rows = [];

+     ImportExcelDataJob::dispatch( $rows );
}

  

Then, to receive our $rows argument in the job class, we pass it to the job’s constructor, and finally process its value in the handle method:

    /* app\Jobs\ImportExceltDataJob  */
public function __construct( protected $rows ){}

public function handle( \App\Http\Services\ExcelRowProcessor $processor )
{
    foreach( $this->rows as $key=> $row )
        $processor->process($key, $row);
}

  

See the $processor parameter? It’s a service class we can create to process our import.

Responding

Now that our data-processing logic is sent to the background queue, we can as easily dispatch an event to the browser from our server like so:

    /* app\Http\Livewire\ExcelImporter  */
public function importData( $rows ){
     $this->rows = $rows;
     $this->validate();
     $this->rows = [];
     ImportExcelDataJob::dispatch( $rows );

+    $this->dispatchBrowserEvent('import-processing');
}

  

We can easily listen to this import-processing event from the view JavaScript like so:

    /* app/resources/views/livewire/excel-importer */
window.addEventListener('import-processing', event => {
  alert('Data has been sent for processing!');
});

  

Reasons For Tradition

Earlier above, we briefly went over the “upload first, import later” tradition of importing spreadsheet data to our servers.

We might have circumvented this tradition today, but we must take note that there is pretty good reason why it has stood the test of time. One of the primary reasons hinge around the processing of large files.

Imagine parsing MB’s of spreadsheet data in the client browser—Aha, that’s going to be pretty laggy. Of course there are some “workarounds” to this:

Web Workers

SheetJS CE recommends the use of web workers to read large files. The idea is to simply enclose our “extract snippet” into a “web worker” object that will run a separate process away from client browser to process the snippet.

Declare a Worker object called mWorker, and enclose it with our “extract” snippet:

    var mWorker = new Worker(URL.createObjectURL(new Blob([`\ 
    // Import SheetJS CE in web worker
    importScripts("https://cdn.sheetjs.com/xlsx-latest/package/dist/xlsx.full.min.js");

    self.addEventListener('message', (e) => {

        // The file we passed is read as "e.data"
        const reader = new FileReader();
        reader.readAsArrayBuffer(e.data);

        // Parse file
        reader.onload = function(event) {
            // Parsed content
            var workbook = XLSX.read(event.target.result);
            event.target.result = null;

            // Go through each sheet
            workbook.SheetNames.forEach(function(sheetName) {
                // Current sheet
                var sheet = workbook.Sheets[sheetName];

                // Include header param to return array of arrays
                var param = {header:1};
                var rows  = XLSX.utils.sheet_to_json(sheet, param);
                rows.shift();
            });
        }

    });

`]) ));

  

Since a web worker is separate from our browser, the configuration/scripts/variables in the browser’s script is not included in the web worker. This means we can’t use Livewire’s @this.import() method from the web worker.

Instead, we’ll send back rows data from the worker to the browser. We can do so through its postMessage interface.

    /* Inside web worker's reader.onload codeblock: */
workbook.SheetNames.forEach(function(sheetName) {
    // Parsed content
    var workbook = XLSX.read(event.target.result);

    // Include header param to return array of arrays
    var param = {header:1};
    var rows  = XLSX.utils.sheet_to_json(sheet, param);
    rows.shift();

+   // Pass a sheet's rows back to browser
+   postMessage({ rows });
});

  

Now that we have our mWorker setup, revise the process() function to pass a selected file to this worker, with the help of its postMessage:

    function process(){
    mWorker.postMessage(fileElement.files[0]);
    return false;
} 

  

Once the mWorker emits postMessage from its process, our browser’s script can listen to this, and add the rows as an item to a list of sheet rows:

    let sheetRows = [];
mWorker.onmessage = function(e) { 
  sheetRows.push( e.data.rows ); 
};

  

Our server has limits on the request size it can receive, and so we can’t simply just send the entire e.data.rows received above. Instead we’ll call another function, batchSend:

    var sheetRows = [];
mWorker.onmessage = function(e) { 
    sheetRows.push( e.data.rows ); 
+   batchSend( sheetRows.length-1 );
};

  

And from there, “batch” the number of rows we send to our importData method likeso:

A batch will have a start and end index, forming a sub range from the sheet’s original array of rows.

    function batchSend( index ){

  // Some truths our batching depends on for this sheet
  var batchSize = 100;
  var rowSize   = sheetRows[index].length;

  // Get batch range
  var start = 0;
  var end =  Math.min( start + batchSize, rowSize );

  // Get sliced rows range for current sheet
  var range = sheetRows[index].slice( start, end );

  // Remove range from our current sheet, since `range` holds it
  sheetRows[index] = sheetRows[index].slice( end, rowSize );

  // Send range to server
  @this.importData( range, index );

}

  

Notice how we update sheets[index] with a smaller version of its array with each range sent to the server? As that range has already been sent to the server, we can remove it from our reference.

Now that the sheet’s first batch of rows has been sent to the server, it’s time to send the next batch in. To do so, we’ll have to pass the current sheet’s $index as parameter to importData() in the component. And include that index in the batch-processed event:

    public function importData( $rows, $sheetIndex )
{
    $this->rows = $rows;
    $this->validate();
    $this->rows = [];

    // Include sheetIndex to the job and service class as well...
    ImportExcelDataJob::dispatch( $rows, $sheetIndex );

+    $this->dispatchBrowserEvent('import-processing',[ 
      'sheetIndex' => $sheetIndex 
    ]);
}

  

Then from our import-processing listener in JavaScript, send in the sheet’s next batch:

    window.addEventListener('import-processing',function(event){
    var sheetIndex = event.detail.sheetIndex;
    if( sheetRows[sheetIndex].length > 0 ){
        batchSend( sheetIndex );
    }else{
        alert( sheetIndex + 'Sheet\'s Data sent for processing!');
    }
});

  

Papa Parse For CSVs

If we’re only accepting CSV files, we can use another package to parse data more efficiently. Papa Parse is a fast CSV parser for JavaScript, and, it can stream our file for reading, significantly reducing memory usage!

    <script src="https://cdnjs.cloudflare.com/ajax/libs/PapaParse/5.4.1/papaparse.min.js" ></script>

function papaParse( file ){
  // Create a sheet in our list
  sheetRows.push([]);

  // Stream Read file
  Papa.parse(file, {
      // Read file row by row through step
      step: function(row) {
          // Always at 0 since CSVs can only have one sheet
          sheetRows[0].push(row);
      },
      complete: function() {
          // Once all rows are added, start batch-sending the rows 
          batchSend( 0 );
      }
  });
}

  

Looking Back on Tradition

Today we learned to import data directly from client to server using JavaScript packages, and Livewire. Along the way, we also learned why we have the “upload first, import later” tradition in importing data. It’s not cool to burden user’s machines with parsing large files.

Of course, there are some ways to help alleviate burden from our client browser—like using web workers, or using stream reading to parse sections of our file at a time.

To upload, or not to upload? Well—after reading through this article, if uploading the file sounds the better route, large files will still cause some 413 errors during upload. But, when there’s a will, there’s a way! Why not try uploading the file, in chunks?

Figuring out SPAs

2023-04-04T00:00:00+00:00

People love their precious SPAs! Host your frontend (and heck, even your backend) globally on Fly.io, and reap the rewards of low ping times.

SPAs always are a bit fragile. Nevermind the game of “where’s the business logic?” (a rant I won’t bother writing), splitting the frontend from the backend makes hosting SPAs harder.

The complications are mostly related to security concerns, such as dealing with CORS and cookie configuration. These often fail with confusing browser errors (on the frontend, where errors go to die - never to be read by a human).

I might be biased, but there’s a lot that can go wrong!

I don’t usually run SPAs myself, so when someone asked about doing it on Fly, I wanted to figure it out to ensure I understood all the issues. If I’m going to complain about SPAs, I better understand them!

Here’s my writeup about what it takes to separate your frontend from your backend. I’m writing in context of Laravel (and Sanctum), but this applies to most frameworks.

The Goal

We want our frontend (The JS making up our SPA) hosted on some url like my-app.fly.dev. The backend will be an API hosted on something like my-app-api.fly.dev.

Off the bat we have some hurdles:

We’re using 2 domains, and Javascript is going to need permission to make HTTP requests to the API domain (via your best frenemy CORS)
We need to handle authentication - in this case, cookie based auth (yes, cookies!) across 2 subdomains of the same root domain
We also need to handle CSRF (cross-site request forgery) tokens to protect against malicious attempts to send data to our backend

So, we need our frontend to be able to authenticate and stay authenticated with our backend. It also needs to securely send data (mainly POST requests) to the API.

Authentication and authorization is often done with JWTs, but Laravel Sanctum provides a way to handle most of this using good-old-fashion cookies.

The Factors

There are the factors making SPA setups a bit painful!

Here’s a quick overview of them:

CORS & AJAX

Browsers don’t just let you send HTTP requests anywhere. By default, they only allow you to make requests to the same domain you see in the URL bar. To make requests to other domains (“Cross Origin”), the other domain needs to allow it. It does this via CORS headers. We need our backend to allow the frontend to send HTTP requests to it by setting the correct CORS headers.

Cookies & Domains

Similarly, cookies can only be set for specific domains. Browsers will ignore/omit cookies set for a different domain than the one a site is currently on. However, our backend and frontend domains may be different. In our case, since we want to make use of cookies, we need a common root domain, which will allow subdomains to share cookies. We’ll use cookies for authentication with the backend - the regular old use case for cookies - but across subdomains. You can’t use cookies across 2 totally different domains - browsers won’t let you.

CSRF

Related to cookies is the need for CSRF tokens with Laravel. Laravel uses these to protect against cross-site request forgery (hey, that’s “CSRF”). We’ll send our CSRF tokens along as another (encrypted) cookie to Laravel.

🙅 Fly.dev

Lastly, fly.dev is on the public suffix list. This has a side effect!

We can’t accomplish what we want using a *.fly.dev subdomain. Browsers won’t let you set cookies for all subdomains of public suffix domain. Instead, we need to use a custom domain in this setup! Besides, would you really want your cookies set to work on any site using the *.fly.dev domain? There’s…a lot of them.

Earlier I gave domains my-app.fly.dev and my-app-api.fly.dev as examples. Well, we can’t use those. Let’s pretend instead we set up a custom domain on Fly.io - we’ll use subdomains www.my-app.com and api.my-app.com.

From here we see how to take care of all of this stuff with the help of Laravel Sanctum.

Sanctum

The basics of installing and using Sanctum are the following few things:

    composer require laravel/sanctum
php artisan vendor:publish \
    --provider="Laravel\Sanctum\SanctumServiceProvider"

# Assuming a DB is setup
php artisan migrate

  

Then comes the interesting stuff.

Middleware

For SPA authentication, add the following to your app/Http/Kernel.php for the API middleware:

    'api' => [
\Laravel\Sanctum\Http\Middleware\EnsureFrontendRequestsAreStateful::class,
\Illuminate\Routing\Middleware\ThrottleRequests::class.':api',
\Illuminate\Routing\Middleware\SubstituteBindings::class,
],

  

This ensures requests are “stateful” for “first party” frontends (ones we have registered). Let’s go ahead and register those.

To do that, we need to tell Sanctum what domains we’ll use with “stateful” (cookie-based) auth via the SANCTUM_STATEFUL_DOMAINS env var.

Check file config/sanctum.php to find how that works:

    return [
    // Other options omitted
    'stateful' => explode(',', env('SANCTUM_STATEFUL_DOMAINS', sprintf(
        '%s%s',
        'localhost,localhost:3000,127.0.0.1,127.0.0.1:8000,::1',
        Sanctum::currentApplicationUrlWithPort()
    ))),
    // ... and so on
];

  

The default configuration tries to set the stateful domains for you, but it’s defaulting to values best for local development.

To try this out with 2 different subdomains, I added the following to my environment (via .env locally and fly.toml for deploying to Fly.io):

    SANCTUM_STATEFUL_DOMAINS="www.my-app.com,api.my-app.com"

  

If your SANCTUM_STATEFUL_DOMAINS are using non-standard ports (anything other than 80 or 443), be sure to include the ports in the hostnames, e.g. api.my-app.com:9000.

Configure CORS

We need to ensure the backend (API) is replying with the correct CORS headers to allow the frontend to send it HTTP requests.

From the Sanctum docs on SPA authentication:

In order to authenticate, your SPA and API must share the same top-level domain. However, they may be placed on different subdomains.

So, we’re going to launch two different Fly apps, which will use 2 different subdomains from the top-level domain my-app.com. (www.my-app.com and api.my-app.com).

Sanctum can help with the CORS headers for us! We just need to set some configuration.

Head to config/cors.php and set supports_credentials to true, and ensure any other routes you use are allowed in the paths array:

    return [
    // Allow ourselves to login by adding a /login route
    'paths' => ['api/*', 'sanctum/csrf-cookie', 'login'],

    // Other options omitted

    // Ensure cors supports auth/csrf headers
    'supports_credentials' => true,
];

  

I added the login route so my frontend could send a POST request to /login on the backend to authenticate (and get returned a valid cookie).

Setting supports_credentials to true sets the correct CORS headers.

Configure Cookies

Cookies will, by default, only work for the current domain being used to access the site. However, we’ll be using 2 domains. Since these subdomains use the same root domain my-app.com, we can make the cookies work for both. (If they were totally different domains, we couldn’t do that).

So, for our multiple subdomains (www.my-app.com and api.my-app.com), we’ll set the SESSION_DOMAIN env var to .my-app.com (with the preceding period).

Check file config/session.php to find where that env var is set:

    return [
    // Other options omitted
    'domain' => env('SESSION_DOMAIN'),
];

  

Configure Environment Variables

So far we have 2 environment variables set:

    SESSION_DOMAIN=".my-app.com"
SANCTUM_STATEFUL_DOMAINS="www.my-app.com,api.my-app.com"

One quick note, however, is that Laravel needs the APP_URL to be set correctly as well. This should include the port used if using a non-standard (80,443) port.

Setting this correctly helps ensure that URLs generated by the code (including cookie domains, if you forget to set SESSION_DOMAIN manually) use the correct domain and protocol (http vs https).

So, all together, the important environment variables are:

    # or maybe api.my-app.com:9000
# if using a non-standard port
APP_URL="api.my-app.com"

SESSION_DOMAIN=".my-app.com"
SANCTUM_STATEFUL_DOMAINS="www.my-app.com,api.my-app.com"

  

Fly.io ❤️ Laravel

Fly your servers close to your users—and keep your SPAs speedy. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

The Code

We need our backend (API) to generate a CSRF token for our frontend to send along anytime it makes HTTP requests to the backend (for authentication or for general API calls).

Since it comes with Laravel out of the box, I’ll assume you use Axios to make HTTP requests from your frontend. It has an option to help here - to have it send along cookies to the backend when making requests. The cookies passed along will both keep you authenticated, but also contain the CSRF token the backend provides us.

To handle this, the first thing to do is send users to a login page. Your frontend should send a request to the backend to get a CSRF token, and then allow a login to happen.

In your application, Axios should be set like so:

    axios.defaults.withCredentials = true;

  

If you are not using Axios to make HTTP requests from your frontend, you should perform the equivalent configuration on your own HTTP client. It’s an underlying XMLHttpRequest feature rather than one specific to Axios.

Once that’s set, if your login form is something like this:

    <form id="login" class="py-12">
    <h2 class="pb-4">Login</h2>
    <div class="mb-4">
        <input type="email" name="email" placeholder="email" class="border rounded p-2">
    </div>
    <div class="mb-4">
        <input type="password" name="password" placeholder="password" class="border rounded p-2">
    </div>
    <div class="mb-4">
        <button type="submit" class="border rounded p-2">Login</button>
    </div>
</form>

  

… then your JavaScript for that can look something like this (only “fancier” because presumably you’ll be using a framework and not just Vanilla Javascript™, like me):

    window.addEventListener('load', (event) => {
    // Get CSRF token on page load
    window.axios.get('https://api.my-app.com/sanctum/csrf-cookie').then(response => {

        // We're now ready to login
        let form = document.querySelector('#login')

        form.addEventListener('submit', (event) => {
            event.preventDefault()

            window.axios.post('http://api.my-app.com/login', new FormData(form))
                .then(request => {
                    getUser()
                }).catch(error => {
                    console.log(error)
                    alert('could not auth')
                })
        });
    });
})

// Hit the API to get a user
function getUser() {
    window.axios.get('http://api.my-app.com/api/user')
        .then(response => {
            console.log(response.data)
            alert('we got our user ' + response.data.email)
        }).catch(error => {
            console.log(error)
            alert('could not get user')
        })
}

  

That’s it!

Sanctum and Axios do a lot of heavy lifting for us. Otherwise we’d have to do things like set up CORS ourself (yuck).

Easy, right? Just make sure everything is configured exactly correctly to get around (or, well, to work alongside with) browser security.

If you do something a little wrong, you’ll break your app. No sweat!

Pro Tip: If you’re going to do an SPA, and split your frontend from your backend, then I suggest using Breeze (with the API scaffolding) to set up authentication. It does a bunch of the manual work we just did for you.

Collecting Time Video Metrics with Livewire

2023-03-22T00:00:00+00:00

Today we’ll use Livewire to communicate video event data to our server. Deploy now on Fly.io, and get your Laravel app running in a jiffy!

Alright drumrolls. Because today, we’ll intercept three user-video interaction events, and use Livewire to easily send event data to our server!

The Problem

There are cases when we’d want to get insight on how users interact with the video files we display in our website pages.

For example. Let’s imagine we have a Laravel website where we share online courses. Each course would have video-lessons users can watch and complete in order to progress through a course.

Wouldn’t it be neat to know which specific time users usually pause lessons in? Or how long does each pause take before the user plays the video again? How about points in time that are frequently visited by the users?

Solution

There are several events a video tag emits which we can hook onto to gain insight on how users interact with various video-lessons. In this article, we’ll demonstrate monitoring user-video interaction data through three events:

pause event - to determine the point in time users usually pauses a lesson
play event - to determine how long pauses endure before lessons are played again
seeked event - to determine the point in time that is usually jumped into for a lesson

Today, we’ll hook onto these events to get relevant data, and use Livewire to easily share data to our server 🌟

Version Notice. This article focuses on Livewire v2, details for Livewire v3 would be different though!

Setting Up The Video Route

We’ll be accessing our video file through an endpoint:

    Route::get('/videos/{id}',
    [\App\Http\Controllers\VideoController::class,'stream']
)->name('videos.stream');

  

Our VideoController class will have a simple stream method:

    public function stream(Request $request, $id){
    // getFileDetails() is a custom method to get our video file!
    $details = $this->getFileDetails( $id );
    $header = [
        'Cache-Control','no-cache, must-revalidate'
    ];
    return response()->file(storage_path($details['path']),$header);
}

  

Now that we have a route to our video file, let’s proceed with creating a Livewire component we can play this video file in.

Setting Up the Livewire Component

Let’s create our Livewire component with php artisan make:livewire video-player. This should create two files: A component, and a view. We can embed this component to any blade view, and even pass parameters to it, likeso:

    @livewire('video-player', ['videoId'=>1])

  

To use videoId, we have to declare a matching public attribute in our Component. We’ll use this later below to create the route to access our video file. First, let’s revise our view.

We’ll add a video tag with no source yet; Instead of immediately getting the source, we’ll let the page render the video tag first. Then, we’ll use Livewire’s wire:init directive to trigger loading the source after rendering:

    <!-- view -->
<div wire:init="initVideo">
  <video wire:ignore
    id="videoPlayer" controls width="500px" height="900px"
  />
</div>

  

Once the page completes loading, the wire:init directive will call the method initVideo(). We can do some initial processing in this method. An example would be to generate a temporary $url to access our video file.

Then we can use Livewire’s dispatchBrowserEvent() method to notify the view’s JavaScript to receive this url by emitting a custom event called init-complete:

    /* Component */
public $videoId;

public function initVideo()

  // As mentioned above, generate url with $videoId
  $url =  URL::temporarySignedRoute(
    'videos.stream', now()->addMinutes(30), ['id' =>  $this->videoId]
  );
  // Notify client to set source
  $this->dispatchBrowserEvent('init-complete', ['url'=>$url]);
}

  

We can listen to this event in our JavaScript, and finally set the source of our video tag:

    <!-- view -->
<script>
  var video = document.getElementById("videoPlayer");
  window.addEventListener('init-complete', event => {
    video.src = event.detail.url;
  });

  

Now that we have our video tag loading our source( neatly after page load ), it’s time to monitor our user interaction with the video!

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!

Deploy your Laravel app! →

Only Time Can Tell

One of the main data we can use to get insight on how our user interacts with our video-lessons is through time data. In this article, we’ll use two main sources for insight on time:

A video tag’s currentTime that points to the time in seconds the video is currently at.
A video event’s timestamp that points to the time in milliseconds the event occurred in.

With these two time variables, we can build insight on our user’s time-based interaction with their video lessons:

When do users often pause the Video-Lesson?

Let’s say we want to listen in on which part of our video-lessons users mostly pause on. We can do so by listening to the pause event triggered whenever the user pauses the video, and get the video tag’s currentTime to determine at what video moment the user paused the video in:

Then we can use Livewire to discreetly send this information to be processed in the server through a custom method, @this.sendPauseData().

    <!-- view -->
<script>
// ... previous listener here

video.addEventListener('pause',function(e){
    // readyState = 4 ensures true pause interaction
    if( video.readyState == 4 )
      @this.sendPauseData( video.currentTime );
});

  

@this.sendPauseData() will trigger a request to call a method in our Livewire component named sendPauseData(). We’ll use this method to share the retrieved currentTime in our server, and ultimately record the “pause” event details.

Doing so let’s us keep a history of pauses for our lesson, paving way for determining “possible” engagement-insights on the lesson: like its average paused video moment.

    /* Component */
public function sendPauseData( $videoCurrentTime ){
    // Do some processing on $videoCurrentTime
}

  

How long do users pause at this Video-Lesson moment?

Since we’re recording different points in time a video gets paused in, why not also add in how long the user paused the lesson? Duration would be: (the time played after pause) - (the time paused).

To get the pause time, we simply use timeStamp from the event variable passed in the pause listener event. Then to get our played time, simply get the timeStamp from the play event listener!

Let’s revise our pause event to include the e.timeStamp value. We’ll also create a global variable to tell us the video has been paused—adding this variable will help us later in listening for “true” play events.

    <!-- view -->
<script>
// ... previous listener here

+   var isPaused = true;// Our video is initially on pause
video.addEventListener('pause',function(e){
    // readyState=4 ensures true pause event interaction
+    if( video.readyState == 4 ){
+       isPaused = true;
+       @this.sendPauseData( video.currentTime, e.timeStamp );
+    }

  

Then in our method from the Livewire component, we’ll have to track this timestamp as a public property $pausedAt so it doesn’t get lost:

    /* Component */
+ public $pausedAt;

public function sendPauseData( $videoCurrentTime, $pausedAtTimeStamp ){
+    $this->pausedAt = $pausedAtTimeStamp;
     // Do some processing on $videoCurrentTime and $pausedAtTimeStamp 
}

  

Next, let’s set up our play event listener. When the user finally plays the video in our view, we can simply send the current timeStamp to our Livewire component:

    <!-- view -->
<script>
// ... previous listeners here
var isPaused = true;
video.addEventListener('play',function(e){
    // Make sure video is prev paused
    if( isPaused ){
      isPaused = false;
      @this.sendPlayData( e.timeStamp );
    }
});

  

And use this value to get the time difference from the $pausedAt attribute previously set:

    /* Component */
public function sendPlayData( $playedAtTimestamp ){ 
    if( $this->pausedAt ){
        $diff = $playedAtTimestamp - $this->pausedAt;
        // Do some processing on this new found duration, $diff
    }
}

  

Which point in time Users usually “seek” to?

There are certain parts of a video that users find really helpful to go back to, we can gain insight on this as well!

Again, we can simply use the currentTime to tell us at which lesson moment the user moved or “seeked” the video time to:

    <!-- view -->
<script>
// ... previous listeners here

video.addEventListener('seeked',function(e){
    console.log('User went to:', video.currentTime);
    @this.sendSeekedData( video.currentTime );
});

  

And in our Livewire component, make our due processing with a matching method:

    /* Component */
public function sendSeekedData( $videoCurrentTime ){
  // Record this vide time please, with a "seekedTo" type
  // to gather historical data
  // that can be basis for determining popular video timestamps!
}

  

Catching Time

Time does fly! And often times it’s a bit hard to catch up to it—for example, we just can’t stop time adding up to the years in our lives. Luckily the same isn’t true for time in video analytics and Livewire.

In this article, we’ve learned about three video events: pause, play, and seek. Listening on these events helped us to get data on three “time-centric” ways users interact with our video-lessons through: the video tag’s currentTime and a video-event’s timeStamp value. Finally, we showcased how easily data from video tags in client browsers can be shared to our server with the help of Livewire.

Of course there’s so much more video events we can listen on and test out, if that’s an interest, reading through the HTMLMediaElement documentation would help for other video analytics implementations.

Time does fly, and we can’t often use it wisely. But, using Livewire to share video events from client browser to our server—now that’s one time-wise way to get time analytics sent to our Laravel applications. 🌟

Laravel Bytes

Never Miss a Webhook

The Bug Problem

The Bug Solution

KISS

Use a Separate Code Base

Reusing Our Code Base

The Spikey Traffic Problem

The Spikey Traffic Solution

Octane

The Availability Problem

The Availability Solution

Scale Out

Auto Shut Down

Deployment

Fly.io ❤️ Laravel

Another Language?

Custom Laravel Pulse Cards

A Custom Requirement: Requests Per Machine

Displaying a New Card

Data to Display

Capturing Metrics with Pulse

Retrieving Pulse Metrics

Fly.io ❤️ Laravel

The Happenings Above

Monitoring Fly Machine Resource Usage with Laravel Pulse

Prerequisites

Focusing on the Servers Card

Servers Card

Capturing Resource Usage with pulse:check

Setting Up pulse:check on a Laravel Fly App

Fly.io ❤️ Laravel

Take Away

Reusable Input Components without Livewire, with Livewire

Character motivation

Building the component

Adding a fluent interface

Fly.io ❤️ Laravel

Making the data reactive with Livewire

Enabling and disabling the input

Displaying errors

Autoscaled Queue Workers

The Goal

Doing the Work

Machines to Run Queue Workers

Creating a Machine

When to Create the Always-On Base VM’s

When to Scale Up

Creating Scaled Queue Workers

Fly.io ❤️ Laravel

Scaling Down

Streaming updates with Livewire and Fly Machines

Prerequisites

Livewire’s wire:stream

Re-assigning Long-Running Requests

Catching Requests

Creating the New Instance

Finding the Tasker-Machine

Replaying to the Tasker-Machine

Replaying to the Main Fly Machine

Applying the Middleware

Look Forward!

Fly.io ❤️ Laravel

Making the CLI and Browser Talk

CLI as User Interface

The Magic

Project Setup

Register Command

API Routes

Browser Registration

Fly.io ❤️ Laravel

That’s It

Other Wacky Ideas

Checkboxes, and Streamed updates with Livewire

What We’ll Do

Set Up

Wiring Checkboxes

Streaming Updates with wire:stream

Fly.io ❤️ Laravel

Streaming with wire:stream

Capturing Resource Usage with `pulse:check`

Setting Up `pulse:check` on a Laravel Fly App

Livewire’s `wire:stream`

The Importance in `:key`

Enter `wire:navigate`

3. `window.location`, and Alpine