From 7ce1112c4e343e7a9bc769bdb9b601c08db617bd Mon Sep 17 00:00:00 2001 From: Sven Date: Mon, 30 Mar 2026 09:05:16 +0200 Subject: [PATCH] docs(readme): document all patch scripts, production deployment, and env vars Adds documentation for 14 previously undocumented patch scripts: - AP threading patches (federation-bridge-base-url, compose-default-checked, mastodon-reply-threading) - Conversations Bluesky patches (cursor-fix, self-filter) - Micropub session token fix - Microsub compose draft guard and AP dispatch (fixed // comment prefix) - Indiekit endpoint URLs protocol fix - Webmention sender hentry syntax fix - Endpoint posts fetch diagnostic Adds Production deployment section covering: - start.example.sh structure and webmention poller architecture - FreeBSD rc.d service script (indiekit.rcd.example) - Prometheus metrics shim (metrics-shim.cjs, port 9209) Extends Setup section with required and key optional env var tables. Co-Authored-By: Claude Sonnet 4.6 --- README.md | 99 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 96 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 3dd9d404..ddf05cf5 100644 --- a/README.md +++ b/README.md @@ -410,6 +410,15 @@ Adds a public `GET /activitypub/api/ap-url?url=` endpoint that resolves a blog p **`patch-endpoint-activitypub-locales.mjs`** Injects German (`de`) locale overrides into `@rmdes/indiekit-endpoint-activitypub` (e.g. "Benachrichtigungen", "Mein Profil"). The package ships only an English locale; this copies and customises it. +**`patch-ap-federation-bridge-base-url.mjs`** +Adds an optional `publicationUrl` parameter to `createFedifyMiddleware()` and `fromExpressRequest()`. When provided, uses the configured canonical URL as the base for all Fedify request objects instead of parsing `req.protocol` + `req.get("host")`. Required because nginx can omit `X-Forwarded-Proto` in some configurations, causing Fedify to see `http://` URLs that don't match its configured `https://` base — it then calls `next()`, the request falls through to auth middleware, and AP inbox/WebFinger clients receive a 302 redirect instead of AP JSON. + +**`patch-ap-compose-default-checked.mjs`** +In the AP reader compose form (`/activitypub/admin/reader/compose`), the upstream code hardcoded `target.defaultChecked` to `true` only when `target.name === "@rick@rmendes.net"` — the upstream developer's own handle, which never matches on this installation. Fix: replaces the hardcoded comparison with `target.defaultChecked = target.checked === true`, so the pre-check state comes from the configured syndication target (which has `checked: true` in `indiekit.config.mjs`). Without this, the AP syndication checkbox is always unchecked in the AP reader compose form and replies written there are never federated. + +**`patch-ap-mastodon-reply-threading.mjs`** +After `POST /api/v1/statuses` (Phanpy/Elk creates a post), the handler intentionally did not insert the new post into `ap_timeline` — it relied on the Eleventy build webhook firing 30–120 s later. If the user replies to their own new post during that window, `findTimelineItemById` returns null, `in_reply_to_id` is silently dropped, and the follow-up reply is classified as a "note" (wrong post type, wrong URL path, no `inReplyTo` in AP output, broken thread on Mastodon). Fix: immediately after `postContent.create()`, inserts a provisional timeline item via `addTimelineItem()` using `$setOnInsert` (idempotent — syndicator's later upsert is a no-op). + ### Conversations **`patch-conversations-collection-guards.mjs`** @@ -418,6 +427,12 @@ Adds null-safety guards to `conversation-items.js` so the endpoint does not cras **`patch-conversations-mastodon-disconnect.mjs`** Patches the conversations endpoint to handle a missing or disconnected Mastodon account gracefully — prevents startup crashes when Mastodon credentials are not configured. +**`patch-conversations-bluesky-cursor-fix.mjs`** +The Bluesky `listNotifications` API paginates backwards in time — using the cursor as a polling marker causes the cursor to drift into the past until the poller fetches no new interactions at all. Fix: removes the `cursor` parameter from `fetchBlueskyNotifications` so every poll fetches the latest page, and clears any stale `bluesky_cursor` from the DB state. Deduplication relies on `platform_id` (unchanged). + +**`patch-conversations-bluesky-self-filter.mjs`** +Self-interactions from the blog's own Bluesky account (likes, reposts, replies) appeared as inbound interactions in the conversations endpoint. Two-pronged fix: (1) the scheduler skips storing any notification whose author handle matches `BLUESKY_IDENTIFIER`; (2) the conversations controller filters out items where the author URL matches the site owner's Bluesky profile URL. + ### Files **`patch-endpoint-files-upload-route.mjs`** @@ -447,6 +462,9 @@ Fixes the `~module/path` resolver in `lightningcss.js` to use `require.resolve() ### Micropub +**`patch-micropub-session-token.mjs`** +The Micropub action controller destructures the session token as `session.token`, but the IndieAuth middleware stores it as `session.access_token`. Result: `token` is `undefined` → `Authorization: Bearer undefined` → file upload via OwnYourSwarm always fails with 401. Fix: changes the destructuring to `const { scope, access_token: token }`. + **`patch-endpoint-micropub-where-note-visibility.mjs`** Defaults OwnYourSwarm `/where` check-in notes to `visibility: unlisted` unless the post explicitly sets a visibility. Prevents accidental public syndication of location check-ins. @@ -461,6 +479,9 @@ Adds AI disclosure field UI (text level, code level, etc.) to the post creation/ **`patch-endpoint-posts-ai-cleanup.mjs`** Removes AI disclosure fields from the post form submission before saving, delegating persistence to the AI block sidecar system. +**`patch-endpoint-posts-fetch-diagnostic.mjs`** +In the two-jail setup the node jail cannot reach `https://blog.giersig.eu` directly; self-referential fetches in `@indiekit/endpoint-posts` fail with `ECONNREFUSED`. Fix: rewrites self-referential fetch URLs to `http://localhost:` using `INTERNAL_FETCH_URL` (or an automatic fallback), and wraps the fetch in a try-catch that logs the URL and response status on failure to make networking problems easier to diagnose. + **`patch-endpoint-posts-uid-lookup.mjs`** Fixes post editing 404s by adding `uid`-based lookup to the micropub source query. Without this, posts older than the first 40 results could not be opened for editing. @@ -479,6 +500,9 @@ Prevents unlisted posts from being re-syndicated via `@indiekit/endpoint-syndica ### Indiekit core +**`patch-indiekit-endpoint-urls-protocol.mjs`** +Endpoint URL resolution in `@indiekit/indiekit/lib/endpoints.js` uses `getUrl(request)` which returns the HTTP protocol (Express sees HTTP from nginx). This causes mixed-content failures in Safari when the frontend tries to load endpoint assets from `http://` URLs on an `https://` page. Fix: prefers `application.url` (the configured HTTPS base URL) over `getUrl(request)` when resolving relative endpoint paths. + **`patch-indiekit-routes-rate-limits.mjs`** Replaces the single blanket rate limiter with separate strict (session/auth) and relaxed (general) limiters so legitimate API traffic is not throttled during normal use. @@ -510,8 +534,11 @@ Adds OPML file upload support to the podroll endpoint. ### Microsub / Reader -// **`patch-microsub-reader-ap-dispatch.mjs`** -Adds Fediverse/ActivityPub detection and dispatch to the Microsub reader so AP profile URLs are routed to the ActivityPub reader rather than the RSS reader. +**`patch-microsub-compose-draft-guard.mjs`** +The Microsub reader compose form was syndicating draft posts. Fix: extracts `post-status` from the request body; when the post is a draft, clears all `mp-syndicate-to` targets before forwarding to Micropub, and forwards `post-status: draft` so the post is saved as a draft. + +**`patch-microsub-reader-ap-dispatch.mjs`** +Three related issues in the Microsub reader's `detectProtocol()` and syndication dispatch: (1) the hardcoded fediverse domain list missed common instances (troet.cafe, hachyderm.io, etc.); (2) same-instance Mastodon URLs weren't detected because the naive hostname check didn't match against the configured target set; (3) likes and reposts of fediverse posts were not dispatched as native AP Like/Announce activities. Fix: extends the fediverse detection list; builds the Mastodon hostname set dynamically from configured syndication targets; dispatches native AP Like/Announce for likes/reposts after the Micropub POST succeeds. Serve-only (not postinstall). **`patch-microsub-feed-discovery.mjs`** Improves feed discovery in `fetchAndParseFeed`: when a bookmarked URL is an HTML page, falls back to `` discovery and a broader set of candidate paths rather than only the fixed short list. @@ -523,6 +550,9 @@ Applies several guards to the listening endpoints: scopes Funkwhale history fetc ### Webmention sender +**`patch-webmention-sender-hentry-syntax.mjs`** +`@rmdes/indiekit-endpoint-webmention-sender` v1.0.8 shipped with a typo: `_html.includes("h-entry"")` — the extra closing quote is a `SyntaxError` that prevents the module from loading at all, so the webmention sender never starts. This patch runs first (alphabetically) and fixes the typo before any other webmention-sender patches apply. + **`patch-webmention-sender-livefetch.mjs`** (v6) Replaces the upstream content-fetching block with a synthetic h-entry builder. Reads stored post properties directly from the MongoDB document (`in-reply-to`, `like-of`, `bookmark-of`, `repost-of`, `syndication`, `content.html`) and constructs a minimal `
` with the appropriate microformat anchor tags. No live page fetch, no nginx dependency, no networking failures. Logs which properties were found per post. Upgrades from any prior version (v1–v5) in-place. @@ -744,7 +774,70 @@ npm install # installs dependencies and runs all postinstall patches npm run serve # runs preflights + patches + starts the server ``` -Environment variables are loaded from `.env` via `dotenv`. See `indiekit.config.mjs` for the full configuration. +Environment variables are loaded from `.env` via `dotenv`. Copy `.env.example` to `.env` and fill in the required values. See `indiekit.config.mjs` for the full configuration. + +### Required environment variables + +| Variable | Purpose | +|---|---| +| `SECRET` | Session signing secret — at least 32 characters | +| `PASSWORD_SECRET` | bcrypt hash of the admin password (`$2a$…`) | +| `MONGO_URL` or `MONGO_USERNAME`+`MONGO_PASSWORD` | MongoDB connection | +| `GH_CONTENT_TOKEN` | GitHub token with write access to the `blog` repo | + +### Key optional variables + +| Variable | Default | Purpose | +|---|---|---| +| `PUBLICATION_URL` | `https://blog.giersig.eu` | Canonical blog URL | +| `INDIEKIT_URL` | same as `PUBLICATION_URL` | Application base URL | +| `REDIS_URL` | — | Redis for AP message queue + KV store (production-required for persistence) | +| `INTERNAL_FETCH_URL` | `http://localhost:PORT` | Direct Indiekit URL for self-fetches, bypassing nginx | +| `INDIEKIT_BIND_HOST` | `0.0.0.0` | Jail IP to bind on (FreeBSD jails don't have loopback) | +| `PORT` | `3000` | Listen port | +| `AP_HANDLE` | `svemagie` | ActivityPub username | +| `AP_ALSO_KNOWN_AS` | — | Old Mastodon account URL for migration alias | +| `AP_LOG_LEVEL` | `info` | Fedify log verbosity | +| `BLUESKY_HANDLE` | — | Bluesky handle for syndication | +| `BLUESKY_PASSWORD` | — | Bluesky app password | +| `WEBMENTION_IO_TOKEN` | — | Webmention.io API token | +| `YOUTUBE_OAUTH_CLIENT_ID` | — | YouTube OAuth client ID for likes sync | +| `YOUTUBE_OAUTH_CLIENT_SECRET` | — | YouTube OAuth client secret | +| `LASTFM_API_KEY` | — | Last.fm API key | +| `LASTFM_USERNAME` | — | Last.fm username | +| `FUNKWHALE_INSTANCE` | — | Funkwhale instance URL | +| `FUNKWHALE_USERNAME` | — | Funkwhale username | +| `FUNKWHALE_TOKEN` | — | Funkwhale API token | + +--- + +## Production deployment + +### Startup script (`start.example.sh`) + +Copy `start.example.sh` to `start.sh` and make it executable. It: + +1. Loads `.env` via `dotenv` +2. Validates required variables (`SECRET`, `PASSWORD_SECRET`, `MONGO_URL` or credentials, `GH_CONTENT_TOKEN`) and exits with a clear error if any are missing +3. Runs all preflight scripts (`scripts/preflight-*.mjs`) in order +4. Runs all patch scripts (`scripts/patch-*.mjs`) in order +5. Starts Indiekit via `node --require ./metrics-shim.cjs node_modules/.bin/indiekit serve` +6. Launches the webmention poller as a background subprocess (polls `WEBMENTION_SENDER_POLL_INTERVAL` seconds; default 300) +7. Traps `EXIT`/`INT`/`TERM` to cleanly shut down the poller when Indiekit exits + +The webmention poller connects **directly to Indiekit** at `INDIEKIT_BIND_HOST:PORT` (not through nginx). nginx returns HTTP 444 for requests with an unrecognised `Host` header; the poller's `Host: ` would match nothing and silently fail. + +### FreeBSD rc.d service (`indiekit.rcd.example`) + +Copy `indiekit.rcd.example` to `/usr/local/etc/rc.d/indiekit` (in the node jail) and enable it with `sysrc indiekit_enable=YES`. The script: + +- Uses `daemon(8)` with `-P -p -o ` so both the supervisor and the Node process are tracked +- Supports a configurable stop timeout (`indiekit_stop_timeout`, default 30 s) before sending SIGKILL +- Supports `service indiekit reload` (sends SIGHUP) for configuration reloads without a full restart + +### Prometheus metrics (`metrics-shim.cjs`) + +A CommonJS preload module that runs a Prometheus scrape endpoint on port 9209 (configurable via `METRICS_BIND_HOST` and `METRICS_PORT`). Loaded at startup via `node --require ./metrics-shim.cjs`. Exposes basic Node.js process metrics for scraping by a Prometheus instance. ---