github.com/plausible-hosting
1
0
Fork 1
mirror of https://github.com/plausible/hosting.git synced 2025-07-01 17:47:29 -05:00
Code Issues Projects Releases Wiki Activity

cleanup

Browse Source
This commit is contained in:
ruslandoga
2023-10-12 21:31:34 +08:00
parent 4ad9963b84
commit c4dc2158f0
1 changed files with 23 additions and 78 deletions
Download Patch File Download Diff File Expand all files Collapse all files

101
guides/backup-restore-clickhouse.md
View File

@ -6,14 +6,15 @@ Guide to backing up and restoring `plausible_events_db`. Based on ["Backup and R
### Plan ### Plan
1. dump contents of the Plausible tables (events, sessions, imported\_\*) to files in a mounted volume 1. dump contents of the Plausible tables to a file
1. load dumps from the mounted volume into `plausible_events_db` 1. optionally copy that file to a new host
1. restore the tables from that file into `plausible_events_db`
--- ---
### Backup ### Backup
1. Add backup config to allow reading backup files in the ClickHouse container 1. Add backup config to allow reading and writing backup files in the ClickHouse container
```console ```console
$ touch clickhouse/backups.xml $ touch clickhouse/backups.xml
@ -21,115 +22,59 @@ $ touch clickhouse/backups.xml
```xml ```xml
<clickhouse> <clickhouse>
<storage_configuration> <backups>
<disks> <allowed_path>/backups/</allowed_path>
<backups> </backups>
<type>local</type>
<path>/backups/</path>
</backups>
</disks>
</storage_configuration>
<backups>
<allowed_disk>backups</allowed_disk>
<allowed_path>/backups/</allowed_path>
</backups>
</clickhouse> </clickhouse>
``` ```
2. Add backups volume to `plausible_events_db` in your `docker-compose.yml` 2. Mount new configuration to volume to `plausible_events_db` in your `docker-compose.yml`
```diff ```diff
plausible_events_db: plausible_events_db:
image: clickhouse/clickhouse-server:22.6 image: clickhouse/clickhouse-server:23-alpine
volumes: volumes:
- event-data:/var/lib/clickhouse - event-data:/var/lib/clickhouse
- ./clickhouse/:/etc/clickhouse-server/config.d/ - ./clickhouse/:/etc/clickhouse-server/config.d/
+ - ./clickhouse-backups:/backups
``` ```
3. Stop relevant containers to avoid writing to old `plausible_events_db` 3. Dump old `plausible_events_db` contents to a backup file
```console ```console
> docker compose stop plausible plausible_events_db > docker compose exec plausible_events_db sh -c "clickhouse-client -q 'BACKUP DATABASE plausible_events_db TO File(\'/backups/plausible_events_db.zip\')'"
[+] Running 2/2
⠿ Container hosting-plausible-1 Stopped 6.5s
⠿ Container hosting-plausible_events_db-1 Stopped 0.2s
``` ```
4. Restart `plausible_events_db` container to attach volume 4. Copy the dump to the host
```console ```console
> docker compose up plausible_events_db -d > docker compose cp plausible_events_db:/backups/plausible_events_db.zip plausible_events_db.zip
[+] Running 1/1
⠿ Container hosting-plausible_events_db-1 Started 0.3s
``` ```
5. Dump old `plausible_events_db` contents to backup files 5. (Optional) verify backup went OK
```console ```console
> docker compose exec plausible_events_db sh -c "clickhouse-client --query 'BACKUP TABLE plausible_events_db.events TO Disk('backups', 'events.zip')'" > unzip -l plausible_events_db.zip
> docker compose exec plausible_events_db sh -c "clickhouse-client --query 'BACKUP TABLE plausible_events_db.sessions TO Disk('backups', 'sessions.zip')'"
> docker compose exec plausible_events_db sh -c "clickhouse-client --query 'BACKUP TABLE plausible_events_db.imported_visitors TO Disk('backups', 'imported_visitors.zip')'"
# etc.
``` ```
**TODO:** script 6. Move the dump somewhere, like a different host
6. (Optional) verify backup went OK
```console
> ls backups
events.zip sessions.zip imported_visitors.zip ...
```
7. Move the backups somewhere, like a different host
### Restore ### Restore
1. Ensure relevant containers are stopped 1. Copy the dump into the container
```console ```console
> docker compose stop plausible plausible_events_db > docker compose cp plausible_events_db.zip plausible_events_dh:/backups/plausible_events_db.zip
[+] Running 2/2
⠿ Container hosting-plausible-1 Stopped 0.0s
⠿ Container hosting-plausible_events_db-1 Stopped 0.2s
``` ```
3. Ensure backup config from step 1 and backups mount from step 2 are still present. 2. Restore
4. Load data into from backups. **TODO:** User + permissions
```console ```console
> docker compose exec plausible_events_db sh -c "clickhouse-client --query 'RESTORE TABLE plausible_events_db.events FROM Disk('backups', 'events.zip')'" > docker compose exec plausible_events_db sh -c "clickhouse-client -q 'RESTORE DATABSE plausible_events_db FROM File(\'/backups/plausible_events_db.zip\')'"
> docker compose exec plausible_events_db sh -c "clickhouse-client --query 'RESTORE TABLE plausible_events_db.sessions FROM Disk('backups', 'sessions.zip')'"
> docker compose exec plausible_events_db sh -c "clickhouse-client --query 'RESTORE TABLE plausible_events_db.imported_visitors FROM Disk('backups', 'imported_visitors.zip')'"
# etc.
``` ```
6. Start all other containers 3. (Optional) Remove backups from the container and the host
```console ```console
> docker compose up -d > docker compose exec plausible_events_db rm -rf /backups/plausible_events_db.zip
[+] Running 4/4 > rm plausible_events_db.zip
⠿ Container hosting-plausible_events_db-1 Running 0.0s
⠿ Container hosting-mail-1 Running 0.0s
⠿ Container hosting-plausible_db-1 Started 0.5s
⠿ Container hosting-plausible-1 Started 0.5s
```
7. (Optional) Remove backups from `docker-compose.yml` and your filesystem
```diff
plausible_events_db:
image: clickhouse/clickhouse-server:22.6
volumes:
- event-data:/var/lib/clickhouse
- ./clickhouse/:/etc/clickhouse-server/config.d/
- - ./clickhouse-backups:/backups
```
```
> rm -rf ./clickhouse-backups
``` ```