Create LICENSE

This reverts commit 43e8842846.

.github/workflows/spellcheck.yml

@@ -0,0 +1,20 @@
+name: spellcheck
+
+on:
+  pull_request:
+  push:
+
+jobs:
+  codespell:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: codespell-project/actions-codespell@v2
+        with:
+          check_filenames: true
+
+  typos:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: crate-ci/typos@master

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Plausible Analytics
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -7,7 +7,7 @@
 </p>
 
 <p align="center">
-    <strong>A getting started guide to self-hosting Plausible Community Edition</strong>
+    <strong>A getting started guide to self-hosting <a href="https://plausible.io/blog/community-edition">Plausible Community Edition</a></strong>
 </p>
 
 <!-- TODO latest version, current version, requirements -->
@@ -31,7 +31,7 @@
 
 ## Install
 
-Plausible Community Edition (or CE for short) is designed to be self-hosted through Docker. You don't have to be a Docker expert to launch your own instance, but you should have a basic understanding of the command-line and networking to successfully set it up.
+[Plausible Community Edition (or CE for short)](https://plausible.io/blog/community-edition) is designed to be self-hosted through [Docker.](https://docs.docker.com/guides/get-started/) You don't have to be a Docker expert to launch your own instance, but you should have a basic understanding of the command-line and networking to successfully set it up.
 
 ### Requirements
 
@@ -61,7 +61,7 @@ README.md           clickhouse/         docker-compose.yml  images/
 
 In the downloaded directory you'll find two important files:
 
-- [docker-compose.yml](./docker-compose.yml) — installs and orchestrates networking between your Plausible CE server, Postgres database, Clickhouse database (for stats), and an SMTP server.
+- [docker-compose.yml](./docker-compose.yml) — installs and orchestrates networking between your Plausible CE server, Postgres database, and Clickhouse database for stats.
 - [plausible-conf.env](./plausible-conf.env) — configures the Plausible server itself. Full configuration options are documented [below.](#configure)
 
 Right now the latter looks like this:
@@ -70,18 +70,21 @@ Right now the latter looks like this:
 ```env
 BASE_URL=replace-me
 SECRET_KEY_BASE=replace-me
+TOTP_VAULT_KEY=replace-me
 ```
 
 Let's do as it asks and populate these required environment variables with our own values.
 
 #### Required configuration
 
-First we generate the [secret key base](#secret_key_base) using OpenSSL:
+First we generate the [secret key base](#secret_key_base) and [TOTP vault key](#totp_vault_key) using OpenSSL:
 
 <sub><kbd>console</kbd></sub>
 ```console
 $ openssl rand -base64 48
 GLVzDZW04FzuS1gMcmBRVhwgd4Gu9YmSl/k/TqfTUXti7FLBd7aflXeQDdwCj6Cz
+$ openssl rand -base64 32
+dsxvbn3jxDd16az2QpsX5B8O+llxjQ2SJE2i5Bzx38I=
 ```
 
 And then we decide on the [base URL](#base_url) where the instance would be accessible:
@@ -92,6 +95,8 @@ And then we decide on the [base URL](#base_url) where the instance would be acce
 + BASE_URL=http://plausible.example.com
 - SECRET_KEY_BASE=replace-me
 + SECRET_KEY_BASE=GLVzDZW04FzuS1gMcmBRVhwgd4Gu9YmSl/k/TqfTUXti7FLBd7aflXeQDdwCj6Cz
+- TOTP_VAULT_KEY=replace-me
++ TOTP_VAULT_KEY=dsxvbn3jxDd16az2QpsX5B8O+llxjQ2SJE2i5Bzx38I=
 ```
 
 We can start our instance now but the requests would be served over HTTP. Not cool! Let's configure [Caddy](https://caddyserver.com) to enable HTTPS.
@@ -142,6 +147,7 @@ Finally we need to update the base URL to use HTTPS scheme.
 - BASE_URL=http://plausible.example.com
 + BASE_URL=https://plausible.example.com
   SECRET_KEY_BASE=GLVzDZW04FzuS1gMcmBRVhwgd4Gu9YmSl/k/TqfTUXti7FLBd7aflXeQDdwCj6Cz
+  TOTP_VAULT_KEY=dsxvbn3jxDd16az2QpsX5B8O+llxjQ2SJE2i5Bzx38I=
 ```
 
 Now we can start everything together.
@@ -175,22 +181,20 @@ Next we'll go over how to upgrade the instance when a new release comes out, mor
 
 ## Upgrade
 
-Each new [release](https://github.com/plausible/analytics/releases/tag/v2.0.0) contains information on how to upgrade to it from the previous version. This section outlines the
+Each new [release](https://github.com/plausible/analytics/releases) contains information on how to upgrade to it from the previous version. This section outlines the general steps and explains the versioning.
-general steps and explains the versioning.
 
 ### Version management
 
 Plausible CE follows [semantic versioning:](https://semver.org/) `MAJOR.MINOR.PATCH`
 
-You can find available Plausible versions on [DockerHub](https://hub.docker.com/r/plausible/analytics). The default `latest` tag refers to the latest stable release tag. You can also pin your version:
+You can find available Plausible versions on [Github packages.](https://github.com/plausible/analytics/pkgs/container/community-edition) The default `latest` tag refers to the latest stable release tag. You can also pin your version:
 
-- <kbd>plausible/analytics:v2</kbd> pins the major version to 2 but allows minor and patch version upgrades
+- <kbd>ghcr.io/plausible/community-edition:v2</kbd> pins the major version to `2` but allows minor and patch version upgrades
-- <kbd>plausible/analytics:v2.0</kbd> pins the minor version to 2.0 but allows only patch upgrades
+- <kbd>ghcr.io/plausible/community-edition:v2.1</kbd> pins the minor version to `2.1` but allows only patch upgrades
 
 None of the functionality is backported to older versions. If you wish to get the latest bug fixes and security updates you need to upgrade to a newer version.
 
-New versions are published on [the releases page](https://github.com/plausible/analytics/releases) and their changes are documented in our [Changelog.](https://github.com/plausible/analytics/blob/master/CHANGELOG.md) Please note that database schema changes require running migrations when you're upgrading. However, we consider the schema
+New versions are published on [the releases page](https://github.com/plausible/analytics/releases) and their changes are documented in our [Changelog.](https://github.com/plausible/analytics/blob/master/CHANGELOG.md) Please note that database schema changes require running migrations when you're upgrading. However, we consider the schema as an internal API and therefore schema changes aren't considered a breaking change.
-as an internal API and therefore schema changes aren't considered a breaking change.
 
 We recommend to pin the major version instead of using `latest`. Either way the general flow for upgrading between minor version would look like this:
 
@@ -219,14 +223,12 @@ $ docker compose -f docker-compose.yml -f reverse-proxy/docker-compose.caddy-gen
  ✔ Container hosting-plausible_db-1         Running    0.0s
  ✔ Container hosting-plausible-1            Started    1.2s
  ✔ Container caddy-gen                      Running    0.0s
-$ docker images --filter=reference='plausible/analytics:*'
+$ docker images --filter=reference='ghcr.io/plausible/community-edition:*'
 REPOSITORY                            TAG           IMAGE ID       CREATED        SIZE
-plausible/analytics   v2.0      2b2735265a65   7 months ago   163MB
+ghcr.io/plausible/community-edition   v2.1          63f7c8708294   6 days ago     83.4MB
-plausible/analytics   v1.5      5e1e0047953a   8 months ago   130MB
+ghcr.io/plausible/community-edition   v2.1.0-rc.0   2b2735265a65   7 months ago   163MB
-$ docker rmi 5e1e0047953a
+$ docker rmi 2b2735265a65
-Untagged: plausible/analytics:v1.5
+Untagged: ghcr.io/plausible/community-edition:v2.1.0-rc.0
-Untagged: plausible/analytics@sha256:365124b00f103ac40ce3c64cd49a869d94f2ded221d9bb7900be1cecfaf34acf
-Deleted: sha256:5e1e0047953afc179ee884389e152b3f07343fb34e5586f9ecc2f33c6ba3bcaa
 ...
 ```
 
@@ -250,6 +252,7 @@ Here's the minimal configuration file we got from the [quick start:](#quick-star
 ```env
 BASE_URL=https://plausible.example.com
 SECRET_KEY_BASE=GLVzDZW04FzuS1gMcmBRVhwgd4Gu9YmSl/k/TqfTUXti7FLBd7aflXeQDdwCj6Cz
+TOTP_VAULT_KEY=dsxvbn3jxDd16az2QpsX5B8O+llxjQ2SJE2i5Bzx38I=
 ```
 
 And here's a configuration with some extra options provided:
@@ -258,12 +261,18 @@ And here's a configuration with some extra options provided:
 ```env
 BASE_URL=https://plausible.example.com
 SECRET_KEY_BASE=GLVzDZW04FzuS1gMcmBRVhwgd4Gu9YmSl/k/TqfTUXti7FLBd7aflXeQDdwCj6Cz
+TOTP_VAULT_KEY=dsxvbn3jxDd16az2QpsX5B8O+llxjQ2SJE2i5Bzx38I=
 MAXMIND_LICENSE_KEY=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
 MAXMIND_EDITION=GeoLite2-City
 GOOGLE_CLIENT_ID=140927866833-002gqg48rl4iku76lbkk0qhu0i0m7bia.apps.googleusercontent.com
 GOOGLE_CLIENT_SECRET=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
 MAILER_NAME=Plausible
-MAILER_EMAIL=plausible@plausible.example.com
+MAILER_EMAIL=somebody+plausible@gmail.com
+MAILER_ADAPTER=Bamboo.Mua
+SMTP_HOST_ADDR=smtp.gmail.com
+SMTP_HOST_PORT=587
+SMTP_USER_NAME=somebody@gmail.com
+SMTP_USER_PWD="wnqj fkbn jcwc byxk" 
 DISABLE_REGISTRATION=invite_only
 ```
 
@@ -303,6 +312,23 @@ SECRET_KEY_BASE=GLVzDZW04FzuS1gMcmBRVhwgd4Gu9YmSl/k/TqfTUXti7FLBd7aflXeQDdwCj6Cz
 > [!WARNING]
 > Don't use this exact value or someone would be able to sign a cookie with `user_id=1` and log in as the admin!
 
+---
+
+#### TOTP_VAULT_KEY
+
+Configures the secret used for encrypting TOTP secrets at rest using AES256-GCM, doesn't have any defaults and needs to be provided in the ENV vars, can be generated with OpenSSL:
+
+<sub><kbd>console</kbd></sub>
+```console
+$ openssl rand -base64 32
+dsxvbn3jxDd16az2QpsX5B8O+llxjQ2SJE2i5Bzx38I=
+```
+
+<sub><kbd>plausible-conf.env</kbd></sub>
+```env
+TOTP_VAULT_KEY=dsxvbn3jxDd16az2QpsX5B8O+llxjQ2SJE2i5Bzx38I=
+``````
+
 ### Registration
 
 #### DISABLE_REGISTRATION
@@ -462,7 +488,9 @@ MaxMind database edition to use (only if [MAXMIND_LICENSE_KEY](#maxmind_license_
 
 Plausible CE sends transactional emails e.g. account activation, password reset. In addition, it sends non-transactional emails like weekly or monthly reports.
 
-It uses SMTP with a [relay](./docker-compose.yml#L3-L5) by default. Alternatively, you can use other [services](https://hexdocs.pm/bamboo/readme.html#available-adapters) such as Postmark, Mailgun, Mandrill or Send Grid to send emails.
+It uses SMTP with an optional [relay](#smtp_host_addr) by default. Alternatively, you can use other [services](https://hexdocs.pm/bamboo/readme.html#available-adapters) such as Postmark, Mailgun, Mandrill or Send Grid to send emails.
+
+[Here's](https://gist.github.com/ruslandoga/c94ce526231fb77930132aaeda3fc3c9) a short guide on using your Gmail account for email delivery.
 
 #### MAILER_ADAPTER
 
@@ -470,6 +498,29 @@ Default: `Bamboo.SMTPAdapter`
 
 Instead of the default, you can replace this with <kbd>Bamboo.PostmarkAdapter</kbd>, <kbd>Bamboo.MailgunAdapter</kbd>, <kbd>Bamboo.MandrillAdapter</kbd> or <kbd>Bamboo.SendGridAdapter</kbd> and add the appropriate variables.
 
+<details>
+<summary>Having problems with the default SMTP client?</summary>
+
+---
+
+Please try the new SMTP client introduced in [v2.1.0](https://github.com/plausible/analytics/discussions/4125) by setting MAILER_ADAPTER to `Bamboo.Mua`. All the `SMTP_*` environment variables can stay the same.
+
+<sub><kbd>plausible-conf.env</kbd></sub>
+```diff
+  BASE_URL=https://plausible.example.com
+  SECRET_KEY_BASE=PkVcxRgQDfQyhPETlog3vvCrj5LdYFSv4ejPEJHJO+i/37w6RZfRjeVCpJayjUjJMfXsNurcv5upPhTRoD3KgQ==
+  TOTP_VAULT_KEY=aihU7k+GSBKbcVFuX9VEPyUhuwlKEomqA94/SQQ0NR4=
+  SMTP_HOST_ADDR=smtp.gmail.com
+  SMTP_HOST_PORT=587
+  SMTP_USER_NAME=somebody@gmail.com
+  SMTP_USER_PWD="wnqj fkbn jcwc byxk"
++ MAILER_ADAPTER=Bamboo.Mua
+```
+
+---
+
+</details>
+
 #### MAILER_EMAIL
 
 Default: `hello@plausible.local`
@@ -484,8 +535,6 @@ The display name for the sender (_from_).
 
 #### SMTP_HOST_ADDR
 
-Default: [`mail`](./docker-compose.yml#L3-L5)
-
 The host address of your SMTP relay.
 
 #### SMTP_HOST_PORT
@@ -575,7 +624,7 @@ Here's the outline of what we'll do:
 - [Integrate with Google Search](#integrate-with-google-search)
   - [Enable APIs for Google Search integration](#enable-apis-for-google-search-integration)
   - [Link it with Plausible](#link-it-with-plausible)
-- [Import historical data from Universal Analytics](#import-historical-data-from-universal-analytics)
+- [Import historical data from UA and GA4](#import-historical-data-from-ua-and-ga4)
   - [Enable APIs for exports on Google Cloud](#enable-apis-for-exports-on-google-cloud)
   - [Import into Plausible](#import-into-plausible)
 
@@ -737,7 +786,7 @@ Go to the site settings on your Plausible dashboard.
 
 <img src="./images/6-plausible-settings-pick.png">
 
-In the settings select <kbd>Search Console</kbd> and press <kbd>Continue with Google</kbd>
+In the settings select <kbd>Integrations</kbd> and press <kbd>Continue with Google</kbd>
 
 > If you see a warning instead, that means you haven't set the <kbd>GOOGLE_CLIENT_ID</kbd> and <kbd>GOOGLE_CLIENT_SECRET</kbd> environment variables [correctly.](#issue-an-oauth-client-and-key-for-that-application)
 
@@ -751,6 +800,10 @@ Trust our own application.
 
 <img src="./images/6-continue.png">
 
+Trust more.
+
+<img src="./images/6-sign-in-copycat.png">
+
 Allow viewing Search Console data.
 
 <img src="./images/6-view-search-console-data.png">
@@ -761,7 +814,13 @@ Pick the property from Search Console.
 
 And now we should be able to drilldown into Google search terms like on [plausible.io](https://plausible.io/plausible.io/referrers/Google?source=Google)
 
-### Import historical data from Universal Analytics
+### Import historical data from UA and GA4
+
+Exporting historical data from Google requires enabling multiple APIs:
+
+<img src="./images/6-enabled-apis.png">
+
+Gotta Catch ‘Em All!
 
 #### Enable APIs for exports on Google Cloud
 
@@ -781,13 +840,31 @@ And enable it.
 
 <img src="./images/7-analytics-reporting-api-enable.png">
 
+Now search for <kbd>Google Analytics Admin API</kbd>
+
+<img src="./images/7-analytics-admin-api-search.png">
+
+And also enable it.
+
+<img src="./images/7-analytics-admin-api-enable.png">
+
+Finally search for <kbd>Google Analytics Data API</kbd>
+
+<img src="./images/7-analytics-data-api-search.png">
+
+And enable it!
+
+<img src="./images/7-analytics-data-api-enable.png">
+
+Phew!
+
 #### Import into Plausible
 
 Go to the site settings on your Plausible dashboard.
 
 <img src="./images/6-plausible-settings-pick.png">
 
-In the <kbd>General</kbd> settings section scroll down to <kbd>Data Import from Google Analytics</kbd> and press <kbd>Continue with Google</kbd> button.
+In the <kbd>Imports & Exports</kbd> settings section press on the <kbd>Google Analytics</kbd> button.
 
 > If you see a warning instead, that means you haven't set the <kbd>GOOGLE_CLIENT_ID</kbd> and <kbd>GOOGLE_CLIENT_SECRET</kbd> environment variables [correctly.](#issue-an-oauth-client-and-key-for-that-application)
 
@@ -801,10 +878,22 @@ Trust our own application.
 
 <img src="./images/6-continue.png">
 
-Pick the view to import and then follow the Plausible directions.
+Trust more.
+
+<img src="./images/6-data-continue.png">
+
+Pick the view to import.
 
 <img src="./images/6-pick-view.png">
 
+And then follow the Plausible directions.
+
+<img src="./images/6-import-continue.png">
+
+Confirm everything's in order and start the import.
+
+<img src="./images/6-import.png">
+
 You'll receive an email once the data is imported.
 
 ---
@@ -813,14 +902,14 @@ You'll receive an email once the data is imported.
 
 ### MaxMind integration
 
-To use MaxMind you need to create an account [here.](https://www.maxmind.com/en/geolite2/signup) Once you have your account details, you can add [MAXMIND_LICENSE_KEY](#maxmind_license_key) and [MAXMIND_EDITION](#maxmind_edition) environmental valiables to your [plausible-conf.env](./plausible-conf.env) and the databases would be automatically downloaded and kept up to date. Note that using city-level databases like MaxMind's GeoLite2-City requires ~1GB more RAM.
+To use MaxMind you need to create an account [here.](https://www.maxmind.com/en/geolite2/signup) Once you have your account details, you can add [MAXMIND_LICENSE_KEY](#maxmind_license_key) and [MAXMIND_EDITION](#maxmind_edition) environmental variables to your [plausible-conf.env](./plausible-conf.env) and the databases would be automatically downloaded and kept up to date. Note that using city-level databases like MaxMind's GeoLite2-City requires ~1GB more RAM.
 
 ## FAQ
 
 <details>
 <summary>How do I access Plausible from terminal?</summary>
 
-You can starts an Interactive Elixir session from within the `plausible` container:
+You can start an Interactive Elixir session from within the `plausible` container:
 
 <sub><kbd>console</kbd></sub>
 ```console
@@ -848,7 +937,7 @@ iex> Application.get_all_env :plausible
 <details>
 <summary>How do I access ClickHouse from terminal?</summary>
 
-You can starts a `clickhouse client` session from within the `plausible_events_db` container:
+You can start a `clickhouse client` session from within the `plausible_events_db` container:
 
 <sub><kbd>console</kbd></sub>
 ```console
@@ -886,7 +975,7 @@ $ docker compose exec plausible_events_db clickhouse client --database plausible
 <details>
 <summary>How do I access PostgreSQL from terminal?</summary>
 
-You can starts a `psql` session from within the `plausible_db` container:
+You can start a `psql` session from within the `plausible_db` container:
 
 <sub><kbd>console</kbd></sub>
 ```console

clickhouse/clickhouse-config.xml

@@ -4,6 +4,13 @@
         <console>true</console>
     </logger>
     
+    <!-- 
+    Avoid the warning: "Listen [::]:9009 failed: Address family for hostname not supported". 
+    If Docker has IPv6 disabled, bind ClickHouse to IPv4 to prevent this issue. 
+    Add this to the configuration to ensure it listens on all IPv4 interfaces:
+    <listen_host>0.0.0.0</listen_host>
+    -->
+
     <!-- Stop all the unnecessary logging -->
     <query_thread_log remove="remove"/>
     <query_log remove="remove"/>

docker-compose.yml

@@ -1,12 +1,8 @@
 services:
-  mail:
-    image: bytemark/smtp
-    restart: always
-
   plausible_db:
-    # Plausible v2.0.0 was tested against PostgreSQL versions 12, 13, and 14
+    # Plausible v2.1.1 was tested against PostgreSQL versions 15 and 16
-    # https://github.com/plausible/analytics/blob/v2.0.0/.github/workflows/elixir.yml#L16
+    # https://github.com/plausible/analytics/blob/v2.1.1/.github/workflows/elixir.yml#L21-L32
-    image: postgres:14-alpine
+    image: postgres:16-alpine
     restart: always
     volumes:
       - db-data:/var/lib/postgresql/data
@@ -14,7 +10,7 @@ services:
       - POSTGRES_PASSWORD=postgres
 
   plausible_events_db:
-    image: clickhouse/clickhouse-server:23.3.7.5-alpine
+    image: clickhouse/clickhouse-server:24.3.3.102-alpine
     restart: always
     volumes:
       - event-data:/var/lib/clickhouse
@@ -27,13 +23,12 @@ services:
         hard: 262144
 
   plausible:
-    image: plausible/analytics:v2.0
+    image: ghcr.io/plausible/community-edition:v2.1.1
     restart: always
     command: sh -c "sleep 10 && /entrypoint.sh db createdb && /entrypoint.sh db migrate && /entrypoint.sh run"
     depends_on:
       - plausible_db
       - plausible_events_db
-      - mail
     ports:
       - 127.0.0.1:8000:8000
     env_file:
@@ -46,4 +41,3 @@ volumes:
     driver: local
   event-logs:
     driver: local
-

plausible-conf.env

@@ -1,2 +1,3 @@
 BASE_URL=replace-me
 SECRET_KEY_BASE=replace-me
+TOTP_VAULT_KEY=replace-me

reverse-proxy/apache2/plausible.conf

@@ -4,6 +4,8 @@
  ServerName example.com
 
  ProxyPreserveHost On
+ ProxyAddHeaders On
+ ProxyPassMatch ^/(live/websocket)$  ws://localhost:8000/$1
  ProxyPass / http://localhost:8000/
  ProxyPassReverse / http://localhost:8000/
 

reverse-proxy/nginx/plausible

@@ -8,8 +8,12 @@ server {
 	location / {
 		proxy_pass http://127.0.0.1:8000;
 		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+
+		location = /live/websocket {
+			proxy_pass http://127.0.0.1:8000;
 			proxy_http_version 1.1;
 			proxy_set_header Upgrade $http_upgrade;
 			proxy_set_header Connection "Upgrade";
 		}
+	}
 }