How a deploy runs

A deploy starts when you push to the configured branch (GitHub calls powder's webhook), or when you redeploy from the dashboard. Powder runs one script on the target server over SSH and streams its output into the deploy log. Each stage prints a banner like ── preflight ──, and when a deploy fails, the dashboard records the name of the last stage that started.

Steps in order

1. app lock. A per-app lock on the server is held for the whole script, so two deploys of one app cannot overlap.
2. preflight. Checks the box, then syncs src/ to the branch on GitHub with the app's read-only deploy key (git sync). Further preflight checks cover the build context and refuse test files placed under pages/ or app/ routes.
3. Build. Either docker compose build on the app server (local build), or, when the server is configured with a dedicated builder host, the image is built there and streamed over (prebuilt image (builder)). Builds are serialized per box and memory-gated.
4. test gate. Optional. If the app ships .powder/test.sh in its build context, it runs in a throwaway Node container against a read-only copy of the code. A non-zero exit stops the deploy and the old container keeps serving. A first-line comment # powder-test-image: <image> overrides the default image.
5. swap gate. A newer deploy of the same app always wins. If an older deploy finishes building after a newer one has started swapping, the swap gate refuses it, so an old commit cannot replace a newer healthy one.
6. db migrate. Runs powder-migrate <app> apply if it is installed on the server. A failing migration aborts the deploy before anything is swapped.
7. release. Optional. If the image has the label powder.release.cmd, that command runs once in a one-off container from the new image before the swap (for example, a schema push). If it fails, the deploy stops and the old container keeps serving.
8. swap-check. Starts the new image as <app>-next and probes it before the live container is touched. See below.
9. docker compose up. Recreates the app's containers from the new image. If the compose file has database or cache services next to the app, only the app service is recreated.
10. health check. Probes the running app. The log shows the target, the number of attempts and which status codes count as healthy.
11. tag. A successful deploy tags the image as <app>:deploy-<id>. These tags are what rollback uses.

swap-check

Before docker compose up, powder starts the new image as a second container called <app>-next and probes it by its container address, using the app's health URL path and the right Host header. It gets 60 seconds, one try every 5 seconds. The candidate is not routed by the proxy and publishes no ports.

The candidate inherits the service's resolved config: env_file and environment:, command:, user:, working_dir, volumes and networks. Volumes are mounted like the real container, so the candidate shares the live app's volumes for the probe window, usually under a minute.

The log lines start with swap-check:.

• healthy in Ns (HTTP 200): the recreate runs and <app>-next is removed.
• FAILED: with a health URL, any non-2xx answer or no answer fails; without one, a real 5xx fails; in both cases an exited candidate fails. The last 200 log lines of the candidate go into the deploy log, the candidate is removed, and the deploy fails at step swap-check. The old container was never touched, so nothing needs rolling back.
• skipped (candidate never answered HTTP): no health URL was set, the candidate never answered over HTTP and is still running. Typical for workers and non-HTTP services. The deploy continues.
• Also skipped, with the deploy continuing: host-network apps, no running container, no curl on the box, or docker compose run failing.

Powder checks every 5 minutes for a candidate container older than 30 minutes on each server. If POWDER_HQ_INGEST_TOKEN and POWDER_HQ_GENERIC_URL are set, it posts a swap-check leftover warning to that endpoint; otherwise nothing is sent. Remove a leftover with docker rm -f <app>-next.

better-auth sign-in probe

After the candidate is healthy, if the app's cloned source mentions better-auth (in package.json or a lockfile, in the app dir or its src/), powder also requests /api/auth/get-session with no cookie. A 5xx fails the swap (swap-check: FAILED (auth check: /api/auth/get-session returned 500 (better-auth schema mismatch? run the migration)), plus the first response line when it mentions SCHEMA_MISMATCH); 2xx, 3xx, 4xx and no answer pass. It tries twice, 3 seconds apart. Per app, set POWDER_AUTH_CHECK in the app's env: off opts out, a path starting with / probes that path and forces the probe without detection. Fleet-wide: POWDER_SWAP_CHECK_AUTH=off in powder.env.

The kill switch

If an app must never have two processes on its volumes at once, turn the check off for that app by setting its swap_check column to false:

UPDATE app_repos SET swap_check = false WHERE container_name = '<app>';

To turn it off for every app on an install, set POWDER_SWAP_CHECK=off in powder.env and restart powder and the worker. Unset means on. When it is off, the log says swap-check: skipped (...) with the reason.

Rollback

Two things protect a deploy after the swap.

• Automatic. After the recreate, powder polls the app. If it does not answer within the timeout and an earlier successful deploy has a tagged image, powder rolls back to it: it restores the env file that image last ran with, retags the image, recreates the container and runs the same health check again. If there is no earlier tagged image, the log says so and leaves the state for you to fix by hand.
• Manual. Use Rollback in the dashboard on any earlier successful deploy. It does the same retag and recreate, which means a few seconds of downtime.

A database migration is not undone by a rollback. Write migrations so that the previous version of the app still works with the new schema, and remove columns only after the code that read them is gone.

Keeping secrets out of images

Apps created with /new and preview deploys (opt-in per app with app_repos.previews_enabled; pushes to non-branch refs and by [bot] pushers never spin one) are cloned into the app directory itself (the older "workdir" layout), so the directory also holds powder's runtime files: .deploy-key, docker-compose.yml and any pgdata/ or kvdata/. To keep them out of image layers, powder writes a .dockerignore next to the compose (an existing one in the repo is kept and topped up). The Dockerfiles powder generates are multi-stage on node:22. On a dedicated builder, powder also writes the file before docker build and refuses to build if a .deploy-key or *.env file sits in the build context unignored.

A repository is untrusted input in this layout. Right after the clone powder refuses to continue if docker-compose.yml, powder.json, .deploy-key, .dockerignore or any .powder-* file is a symlink, and it installs the compose and key through a temp file and mv -f, so a committed link can never redirect a root-run write. Moving these apps to the src/ layout (runtime files outside the repo tree) is the follow-up. It needs readPowderConfig to read src/powder.json first.

Redirects and powder.json

An optional powder.json in the repository can declare redirects, rewrites and response headers. No file in the repository is required: a Dockerfile is enough.

Published ports

Powder publishes ports on 127.0.0.1 only. It never edits your compose file: it writes its own generated docker-compose.powder.yml under /var/lib/powder/compose/<app key>/ (outside any checkout, do not edit; passed by absolute path as the last -f to every docker compose call; a repo that tracks a file with that name is refused) holding the Traefik labels and the loopback bindings. For the app service and for database/cache siblings (postgres, pgvector, mysql, mariadb, redis, valkey, memcached, mongo, clickhouse, elasticsearch, rabbitmq, minio images) every published port that binds all interfaces is replaced by the same port on 127.0.0.1, using the compose !override tag, which needs docker compose 2.24 or newer (older: the deploy log says left public: and nothing is changed). Proxies (nginx, caddy, traefik, haproxy) and other services are left as written and listed as left public: in the deploy log and as a warning in HQ. To keep a service's ports public, give it the label powder.public-port=true in your compose file; powder reads it from the resolved config. The deploy log shows a compose override step with the result.

Rotating DATABASE_URL

The db migrate step reads DATABASE_URL from what the new container will get: the env_file: entries of the compose service in order (last wins), then its environment: overrides, resolved with docker compose config. It does not ask the running container, which still holds the old value until the swap. So a rotated DATABASE_URL (or database password) takes effect at the next deploy's migrate step. Only when no env file or environment: declares it does the step fall back to the running container, and it says so in the log.