diff --git a/.env.example b/.env.example index aba8d3e..c1e2cdf 100644 --- a/.env.example +++ b/.env.example @@ -32,3 +32,13 @@ PGADMIN_DEFAULT_PASSWORD=replace-with-a-third-independent-random-password # Optional image override. Keep the same major version when upgrading an existing volume. POSTGRES_IMAGE=postgres:17.11-alpine + +# Where the two big data sets live. Leave both unset to keep them in Docker volumes, under +# /var/lib/docker. To use a larger disk, give an absolute path to a directory ON it - a +# subdirectory, never the mount point itself, which on ext4 holds lost+found and makes PostgreSQL +# refuse to start. Create the directories and set their owner first; see "Storing the data on +# another disk" in the README. +# POSTGRES_DATA_PATH=/mnt/bigdisk/postgres owner 70:70, mode 700 +# MINIO_DATA_PATH=/mnt/bigdisk/minio owner 1000:1000 +POSTGRES_DATA_PATH= +MINIO_DATA_PATH= diff --git a/README.md b/README.md index babbc6e..c0b4363 100644 --- a/README.md +++ b/README.md @@ -93,6 +93,53 @@ intended. Upgrade PostgreSQL one major version at a time using the PostgreSQL upgrade procedure. Updating an image tag alone does not migrate an existing database volume. +## Storing the data on another disk + +By default PostgreSQL and MinIO keep their data in the Docker volumes `postgres-data` and +`minio-data`, which live under `/var/lib/docker` on the system disk. To use a larger disk, mount it +on the host and point the two variables at directories on it: + +```env +POSTGRES_DATA_PATH=/mnt/bigdisk/postgres +MINIO_DATA_PATH=/mnt/bigdisk/minio +``` + +The two are independent: set one or both. Unset (or empty) means the named volume, as before. + +1. Create the directories **inside** the disk, not at its mount point, and give them to the user the + container runs as. On ext4 the mount point holds `lost+found`, and PostgreSQL will not start in + a directory that is not empty. + + ```sh + sudo mkdir -p /mnt/bigdisk/postgres /mnt/bigdisk/minio + sudo chown 70:70 /mnt/bigdisk/postgres && sudo chmod 700 /mnt/bigdisk/postgres # postgres (alpine) + sudo chown 1000:1000 /mnt/bigdisk/minio # MinIO, see minio.Dockerfile + ``` + +2. **A new installation** needs nothing more: set the variables and `docker compose up -d --build`. + +3. **An installation that already holds data** must copy it first. Do not use `-v` anywhere: + + ```sh + docker compose down + docker run --rm -v humainflow-data-stack_postgres-data:/from -v /mnt/bigdisk/postgres:/to alpine cp -a /from/. /to/ + docker run --rm -v humainflow-data-stack_minio-data:/from -v /mnt/bigdisk/minio:/to alpine cp -a /from/. /to/ + # set the two variables in .env, then + docker compose up -d --build + ``` + + The volume names carry the project name from `name:` in the compose file. Keep the old volumes + until everything works, and remove them only then. + +Cautions: + +- Use a local or block disk. PostgreSQL on a network filesystem such as NFS risks corruption. +- Mount the disk at boot (`/etc/fstab`, with `nofail`) and start Docker after it. If the disk is + missing when the stack starts, Docker creates the directory on the system disk and PostgreSQL + starts **empty** - it looks like the data is gone, and it is only somewhere else. +- A backup of `POSTGRES_DATA_PATH` taken while PostgreSQL runs is not consistent; use `pg_dump` or a + proper base backup. + ## Upgrading from the four-name layout An earlier version used `POSTGRES_DOMAIN`, `MINIO_API_DOMAIN`, `MINIO_CONSOLE_DOMAIN` and diff --git a/docker-compose.yml b/docker-compose.yml index c48a0cf..e7b339a 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -20,7 +20,8 @@ services: - -c - password_encryption=scram-sha-256 volumes: - - postgres-data:/var/lib/postgresql/data + # A host directory when POSTGRES_DATA_PATH is set (see the README), the named volume when not. + - ${POSTGRES_DATA_PATH:-postgres-data}:/var/lib/postgresql/data expose: - "5432" networks: @@ -49,7 +50,8 @@ services: MINIO_BROWSER_REDIRECT_URL: https://${DATA_DOMAIN}:${CONSOLE_PUBLIC_PORT:-9443} command: ["server", "/data", "--console-address", ":9001"] volumes: - - minio-data:/data + # A host directory when MINIO_DATA_PATH is set (see the README), the named volume when not. + - ${MINIO_DATA_PATH:-minio-data}:/data expose: - "9000" - "9001"