feat: gluetun compose and deploy docs #8

Merged

TimCane merged 3 commits from feat/vpn-and-deploy into main

2026-07-19 15:32:40 +01:00

TimCane commented

2026-07-19 15:28:07 +01:00

(Migrated from github.com)

Summary

docker-compose.yaml for the deployment stack: gluetun, Postgres, scraper, MCP server
Dockerfiles for both hosts - multi-stage, manifests restored separately so a code-only change reuses the layer, non-root at runtime
.env.example documenting every value, with .env and *.conf gitignored so a downloaded WireGuard file cannot be committed either
crawl <source> [--max N] / normalize <source> one-shot commands, so a first run against a live source can be capped
README: running the stack, a capped first crawl, connecting an MCP client, and what adding a source actually requires

The VPN wiring

The scraper uses network_mode: "service:gluetun", so every request it makes leaves through the tunnel or not at all - if the tunnel drops it has no network, rather than quietly falling back to the real IP. The MCP server stays on the bridge network and never touches the VPN.

Two things that are easy to get wrong and would fail confusingly:

The killswitch also blocks Postgres. Postgres is on the local docker network, which gluetun's firewall drops by default, so the scraper could not reach its own database. FIREWALL_OUTBOUND_SUBNETS allows that range through
The subnet is pinned. A compose-generated subnet would change between environments and silently break the rule above, so backend has an explicit ipam config that FIREWALL_OUTBOUND_SUBNETS matches
The scraper cannot publish its own ports while sharing a namespace, so 5206 is published by gluetun

Deviation from the earlier plan

We agreed gluetun would sit behind a compose profile in the devcontainer compose. It ended up in the deploy compose without a profile instead, because the scraper and MCP containers only exist there, and the VPN is not optional for the scraper - a profile would make it look like it were. The devcontainer stays app-plus-Postgres and needs no credentials.

Test plan

109 tests pass (64 unit, 45 integration)
docker compose config validates; verified the resolved topology - scraper on service:gluetun with no ports of its own, 5206 published by gluetun, 5205 by mcp, and FIREWALL_OUTBOUND_SUBNETS matching the backend subnet
Both images actually build (352 MB and 354 MB)
Ran the one-shot command against an unknown source to confirm arguments parse and it exits 1 on failure rather than reporting success

No live crawl has been run yet - see below.

## Summary - `docker-compose.yaml` for the deployment stack: gluetun, Postgres, scraper, MCP server - Dockerfiles for both hosts - multi-stage, manifests restored separately so a code-only change reuses the layer, non-root at runtime - `.env.example` documenting every value, with `.env` and `*.conf` gitignored so a downloaded WireGuard file cannot be committed either - `crawl <source> [--max N]` / `normalize <source>` one-shot commands, so a first run against a live source can be capped - README: running the stack, a capped first crawl, connecting an MCP client, and what adding a source actually requires ## The VPN wiring The scraper uses `network_mode: "service:gluetun"`, so every request it makes leaves through the tunnel or not at all - if the tunnel drops it has no network, rather than quietly falling back to the real IP. The MCP server stays on the bridge network and never touches the VPN. Two things that are easy to get wrong and would fail confusingly: - **The killswitch also blocks Postgres.** Postgres is on the local docker network, which gluetun's firewall drops by default, so the scraper could not reach its own database. `FIREWALL_OUTBOUND_SUBNETS` allows that range through - **The subnet is pinned.** A compose-generated subnet would change between environments and silently break the rule above, so `backend` has an explicit `ipam` config that `FIREWALL_OUTBOUND_SUBNETS` matches - The scraper cannot publish its own ports while sharing a namespace, so 5206 is published by gluetun ## Deviation from the earlier plan We agreed gluetun would sit behind a compose profile in the *devcontainer* compose. It ended up in the deploy compose without a profile instead, because the scraper and MCP containers only exist there, and the VPN is not optional for the scraper - a profile would make it look like it were. The devcontainer stays app-plus-Postgres and needs no credentials. ## Test plan - 109 tests pass (64 unit, 45 integration) - `docker compose config` validates; verified the resolved topology - scraper on `service:gluetun` with no ports of its own, 5206 published by gluetun, 5205 by mcp, and `FIREWALL_OUTBOUND_SUBNETS` matching the `backend` subnet - **Both images actually build** (352 MB and 354 MB) - Ran the one-shot command against an unknown source to confirm arguments parse and it exits 1 on failure rather than reporting success No live crawl has been run yet - see below.