readest/docker
Huang Xin 5d2a939c28
fix(storage): check the quota after reserving the row and keep clients off files writes (#6820)
* fix(storage): reserve upload quota atomically and cap public image uploads

Check the storage quota and record the file row in one database call
(reserve_storage_upload, migration 029) that serialises a user's
reservations with an advisory lock and sums live file sizes under it, so
concurrent upload requests can no longer each pass against the same usage.
A reservation is released if signing the upload URL fails.

Require a positive integer fileSize on every upload path, and cap the
unmetered public cover/temp image uploads at 20 MiB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(storage): report re-upload usage without double counting

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(storage): check the quota after reserving the row (owner: optimize the upload quota check instead of adding another layer)

The `files` row an upload writes is already its reservation, and the
trigger on `files` adds its size to `plans.storage_usage_bytes` in the
same transaction. So instead of migration 029's locked RPC, the upload
route now inserts the row first, reads the counter, and deletes the row
and returns 403 if it is over quota. Every accepted request checked after
its own row committed, so the last one to check counted all the others:
concurrent uploads can no longer pass against the same usage. A row is
also dropped if signing fails, and a re-upload of an existing key adds
nothing and is not re-checked. The fileSize validation and the 20 MiB
cap on cover and temp uploads stay.

Share import gets the same check. A fresh import used the token's frozen
usage claim, and restoring a soft-deleted copy was never checked at all;
both now check the counter once the row is live and undo it if over.

Migration 029 now revokes INSERT/UPDATE/DELETE/TRUNCATE on `files` from
anon and authenticated and adds CHECK (file_size >= 0). Only the API
routes write `files`, through the service role, but the grants let a
signed-in user write their own rows through the REST API directly:
insert a negative size, edit a size, or delete a row and keep its object.
init/schema.sql matches it for new installs.

Verified:
- storage-upload-quota.test.ts and share-import-quota.test.ts run
  against main's routes fail 12 and 2 cases: three concurrent uploads
  reach 608174080 bytes against a 524289000 limit; fileSize -1 and
  -107374182400 return 200 and leave a row; a failed signing leaves its
  row; a re-upload is counted twice; over-quota share imports and
  restores return 200 instead of 402. All pass with this change.
- Migration 029 applied twice to a local Postgres 15 `files` table with
  prod's policies and ALL granted to anon/authenticated: before it, a
  signed-in user set their own row's file_size to -5; after it, their
  insert, update and delete fail with "permission denied for table
  files", their select still works, and the service role cannot insert
  a negative size.
- pnpm test (1167 files passed), pnpm lint, pnpm format:check.
- Not run: an end-to-end upload in the app, or a concurrent burst
  against a real database.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(storage): fail closed when the storage usage read fails

Both quota checks fell back to the token's frozen usage claim on any
error reading `plans`, so a transient database error let an upload or
share import pass on stale usage. Only a missing row (PGRST116) or a
missing `plans` table, as on self-hosted servers (PGRST205, or 42P01 from
older PostgREST), now falls back. Any other error returns 500 and undoes
the reservation: the upload deletes its new row, and share import
deletes a fresh row or soft-deletes a restored one again.

Raised by CodeRabbit on #6820.

Verified: the new cases in storage-upload-quota.test.ts and
share-import-quota.test.ts ("fails closed ... when the usage read
fails") fail on the previous head with "expected 200 to be 500" and pass
now; the fallback cases cover all three codes. pnpm test (1167 files
passed), pnpm lint, pnpm format:check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 16:29:27 +02:00
..
volumes fix(storage): check the quota after reserving the row and keep clients off files writes (#6820) 2026-10-20 16:29:27 +02:00
.env.example feat(payments): separate storage and customization entitlements, unlock all when self-hosted (#5996) 2026-09-01 07:37:42 +02:00
compose.build.yaml feat(docker): add GHCR and Docker Hub image publishing (#4250) 2026-05-21 19:57:35 +02:00
compose.dev.yaml fix(docker): fix Docker image latest tag and production runtime errors; add dev compose file, Codespace support, and semver release tagging (#7) (#4277) 2026-05-23 10:39:15 +08:00
compose.yaml docker: pull minio images from quay.io, Docker Hub removed them (#6196) 2026-09-16 17:27:50 +02:00
nginx.conf.example fix(docker): apply db migrations on first boot and let the font CDN be overridden (#5550) (#5551) 2026-08-07 10:19:44 +02:00
README.md docker: pull minio images from quay.io, Docker Hub removed them (#6196) 2026-09-16 17:27:50 +02:00

Self-Hosting with Docker/Podman with Compose

Stack

service Image Description
client ghcr.io/readest/readest readest frontend
db supabase/postgres psql db with supabase extensions
kong kong:2.8.1 api gateway routing requests to supabase services
auth supabase/gotrue:v2.185.0 auth service (email, JWT)
rest postgrest/postgrest:v14.3 psql rest api
minio quay.io/minio/minio s3 storage
minio-setup quay.io/minio/mc helper container to create s3 buckets

Exposed ports

Port Service
3000 readest
8000 kong API gateway
9000 MinIO S3 API
9001 MinIO console UI

Running with Docker/Podman Compose

1. setup .env

cp docker/.env.example docker/.env

update docker/.env:

  • update POSTGRES_PASSWORD to a strong password (32+ chars)
  • update JWT_SECRET to a random secret (32+ chars)
  • regenerate ANON_KEY and SERVICE_ROLE_KEY as HS256 JWTs signed with your JWT_SECRET (use jwt.io or a similar tool):
    • ANON_KEY payload: {"role": "anon"}
    • SERVICE_ROLE_KEY payload: {"role": "service_role"}
  • set MINIO_ROOT_PASSWORD to a strong password

2. Start the Stack (pull prebuilt client image)

run from the docker/ directory:

cd docker
docker compose up -d

this pulls ${READEST_IMAGE} (default: ghcr.io/readest/readest:latest) instead of building the client locally. the web client now reads SUPABASE_PUBLIC_URL, SUPABASE_ANON_KEY, API_BASE_URL, OBJECT_STORAGE_TYPE, SELF_HOSTED, STORAGE_FIXED_QUOTA, and TRANSLATION_FIXED_QUOTA from runtime container env, so custom self-hosted values work with pulled images.

SELF_HOSTED unlocks every premium feature — third-party cloud sync (WebDAV, Google Drive, S3, OneDrive), offline Read Aloud downloads, Send to Readest — with or without a signed-in user. the image defaults it to true, so a pulled image is unlocked even if your compose.yaml predates the variable. set SELF_HOSTED=false in docker/.env only if you run a hosted service that sells the plans.

if you prefer Docker Hub, set READEST_IMAGE in docker/.env, for example:

READEST_IMAGE=docker.io/your-dockerhub-username/readest:latest

replace your-dockerhub-username with the Docker Hub namespace that publishes your readest image. for official images, use the namespace configured for this repository's Docker Hub publishing secrets.

published tags:

  • latest: rolling image from the default branch and from release events
  • <release-tag> (for example v1.2.3): published from release events
  • main: rolling image from the default branch
  • sha-<commit>: immutable commit tag

Build locally instead of pulling

Prerequisites for local builds: the packages/foliate-js and packages/simplecc-wasm git submodules must be initialized before building:

git submodule update --init packages/foliate-js packages/simplecc-wasm

In GitHub Codespaces this is done automatically via .devcontainer/devcontainer.json.

cd docker
docker compose -f compose.yaml -f compose.build.yaml up --build -d

3. Access

  • Readest app: http://localhost:3000
  • MinIO console: http://localhost:9001 (login with MINIO_ROOT_USER / MINIO_ROOT_PASSWORD)

Upgrading an existing deployment

pulling a newer client image does not touch the database volume, and the first-boot hook only runs when that volume is empty. so after an upgrade, apply any new migrations yourself:

cd docker
docker compose pull
docker compose up -d
docker compose exec db /docker-entrypoint-initdb.d/zz-readest-migrations.sh

the script records what it applied in readest_meta.migrations and skips those next time, so it is safe to repeat after every upgrade.

if you had previously patched your database by hand, the script may stop on an already exists error. record that file as applied and run it again:

docker compose exec db psql -U supabase_admin -c \
  "INSERT INTO readest_meta.migrations (name) VALUES ('002_add_book_shares.sql') ON CONFLICT DO NOTHING"

Hot Reload (development)

Prerequisites: submodules must be initialized (see above).

to develop using the compose stack, use compose.dev.yaml which sets the build target to development-stage (Next.js dev server) and mounts your local repo for hot reload:

cd docker
docker compose -f compose.yaml -f compose.dev.yaml up --build -d

the first mount overlays your local repo into the container. the remaining anonymous volumes shadow the directories that were pre-built inside the image, so the container's installed deps and vendor assets are used instead of what's on your host.

Stop the Stack

cd docker
docker compose down

to also remove volumes (database and storage data):

cd docker
docker compose down -v

Database schema

path role
volumes/db/init/schema.sql base schema (books, book_configs, book_notes, files)
volumes/db/migrations/*.sql every schema change since, applied in filename order
volumes/db/apply-migrations.sh applies the migrations and records them in readest_meta.migrations

on an empty database volume the supabase image runs everything under /docker-entrypoint-initdb.d in glob order: its own migrate.sh (supabase core schema plus init-scripts/100-schema.sql, which is schema.sql), then zz-readest-migrations.sh, which is apply-migrations.sh. it globs the mounted migrations directory, so adding a migration file needs no compose change.


Serving from a custom domain

the browser talks to three of these services directly, so each needs a URL that resolves from outside the docker network:

variable what the browser uses it for
SITE_URL the readest client itself
SUPABASE_PUBLIC_URL kong, which routes /auth/v1/… and /rest/v1/…
S3_PUBLIC_ENDPOINT minio, reached through path-style presigned URLs

SUPABASE_PUBLIC_URL and S3_PUBLIC_ENDPOINT default to http://${HOST_IP}:<port>, which suits a plain IP/port deployment; set them in docker/.env to override that. putting everything on one origin also means no cross-origin requests at all:

HOST_IP=your-domain.com
SITE_URL=https://your-domain.com
API_EXTERNAL_URL=https://your-domain.com
ADDITIONAL_REDIRECT_URLS=https://your-domain.com/**
SUPABASE_PUBLIC_URL=https://your-domain.com
S3_PUBLIC_ENDPOINT=https://your-domain.com

nginx.conf.example is a working starting point for terminating TLS in front of the stack. two things it gets right that are easy to miss: the Host header has to reach minio unchanged or the presigned signatures will not verify, and the request body limit has to be lifted on the bucket location or large book uploads are truncated.

CJK fonts on a custom domain

the reader loads a few CJK webfont bundles from Readest's CDN, which only sends Access-Control-Allow-Origin for readest.com origins, so the browser blocks them on a self-hosted domain. mirror https://storage.readest.com/public/font/dist/<Family>/ (and the .woff2 files it references) onto a path your proxy serves, then point the client at it:

FONT_BASE_URL=https://your-domain.com/fonts

leaving FONT_BASE_URL empty keeps the default CDN. system and Google fonts are unaffected either way.


Building the Dockerfile standalone

docker build \
  --target production-stage \
  --build-arg NEXT_PUBLIC_APP_PLATFORM=web \
  -t readest-client \
  .

run the built image:

docker run -p 3000:3000 \
  -e SUPABASE_URL=http://host.docker.internal:8000 \
  -e SUPABASE_PUBLIC_URL=http://localhost:8000 \
  -e SUPABASE_ANON_KEY=<anon-key> \
  -e SUPABASE_ADMIN_KEY=<service-role-key> \
  -e API_BASE_URL=http://localhost:3000 \
  -e OBJECT_STORAGE_TYPE=s3 \
  -e S3_ENDPOINT=http://host.docker.internal:9000 \
  -e S3_PUBLIC_ENDPOINT=http://localhost:9000 \
  -e S3_REGION=us-east-1 \
  -e S3_BUCKET_NAME=readest-files \
  -e S3_ACCESS_KEY_ID=<minio-user> \
  -e S3_SECRET_ACCESS_KEY=<minio-password> \
  -e STORAGE_FIXED_QUOTA=1073741824 \
  -e TRANSLATION_FIXED_QUOTA=50000 \
  readest-client

on Linux, some Docker setups do not resolve host.docker.internal by default. in that case, either replace it with your host IP or run with: --add-host=host.docker.internal:host-gateway.