A preview nobody opens should cost nothing

The first version of Canary billed for every hour a preview existed. It was easy to build and easy to explain, and it was wrong. A typical pull request is open for a day or so. Someone looks at its preview for a few minutes at a time, a handful of times. Charging for the other twenty hours is charging for a server to wait.

So we changed the unit. A preview-hour is an hour one preview is awake. This post is about what "awake" means, what happens when a preview is not, and why the wake has to be fast enough that nobody notices it.

What sleeping keeps and what it stops

preview:
  sleep_after: 30m

After 30 minutes with no requests and no running jobs, the runner puts the preview to sleep. The distinction that matters is between compute and state.

Stopped: every container. Web, API, worker and the database process all exit cleanly. Nothing is using CPU or memory in your cloud account.
Kept: volumes, so the seeded database and anything a reviewer changed are still there. DNS, so the link still resolves. The built images, so nothing rebuilds. Secrets and config, so the preview comes back as the same environment.

A sleeping preview still costs you a few cents of disk in your own cloud. It costs nothing on your Canary bill.

The four-second wake

Sleep only works if waking is boring. If a designer clicks a link and waits a minute, they will ask engineering to turn sleep off, and they will be right.

When a request arrives for a sleeping preview, the runner's proxy holds the connection and starts the containers in dependency order. The request is released when health checks pass. The median is about 4 seconds, and three things got us there.

Images stay on the node. There is no pull on wake. A preview that fell asleep on a node wakes on it unless that node is gone.
The database starts from a clean shutdown. We stop Postgres with a checkpoint instead of killing it, so there is no crash recovery when it comes back. This alone took the wake from around 11 seconds to under 5.
The proxy holds, it does not redirect. The browser sees one slow response, not an interstitial. Webhooks and API clients with ordinary timeouts get their answer without retrying.

The first request after a long sleep is slower than the hundredth. We do not hide that. The preview's activity log shows every wake with its duration.

What counts as a preview-hour

Metering is per second and billed by the minute. The clock runs from the moment a wake begins until the preview goes back to sleep, including the 30 idle minutes at the end. Build time on the first push counts. Time asleep does not, and neither does teardown.

Things that keep a preview awake: HTTP requests, an open WebSocket, a job the worker is still running, and a canary logs session that is following output. Things that do not: our own health checks, and the pull request simply being open.

If a scheduled job needs to run in previews, it will wake the preview when it fires. Most teams switch cron off in previews for exactly that reason.

How much of a preview's life is sleep

Oxbow makes payroll software for independent restaurants, with 64 engineers and a Helm chart on GCP. Before Canary they kept two staging clusters up all month for $2,140. Their previews now spend 86% of their time asleep, and their average bill is $312.

That 86% is not unusual. Across workspaces, a typical preview is awake for a little over three hours of its life. Awake time clusters around the push, when the author checks it, the first review, and the hour before merge. Nights and weekends are almost entirely sleep.

This is also why a spend cap is easy to reason about. Daniel's guide to sizing a spend cap works through the arithmetic.

When to change the default

Thirty minutes is the default because it outlasts a coffee break and not much else. There are reasons to change it.

Shorter, such as 10m, if previews are mostly opened by automated checks that make one request and leave.
Longer, such as 2h, for a demo branch someone keeps returning to over an afternoon. Set it for that one branch in the branch's own canary.yml.
Never is allowed, and it is billed as it sounds. If you need an environment that is always up, that is a staging server, and we would rather say so than charge you for a preview pretending to be one.

The principle underneath all of this is that the bill should follow attention. A preview nobody opens has done nothing for anyone. It should cost nothing, and it should be gone the minute the branch merges. Rowan covers that half in teardown is a feature.