diff --git a/CLAUDE.md b/CLAUDE.md index 57d12624..88d5f8c1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,124 +1,160 @@ # CLAUDE.md — indiekit-blog Personal [Indiekit](https://getindiekit.com/) deployment for [blog.giersig.eu](https://blog.giersig.eu). -Built on the [rmdes/indiekit](https://github.com/rmdes/indiekit) fork ecosystem with a set of -patch scripts that fix issues that cannot be upstreamed. -## Memory files +## Always read memory files first -Detailed architecture and lessons learned live in `memory/`: +Before investigating or modifying anything: -- [`memory/project_activitypub.md`](memory/project_activitypub.md) — AP data flows, own-post timeline, compose paths, all AP patches -- [`memory/project_architecture.md`](memory/project_architecture.md) — FreeBSD jails, nginx, MongoDB collections, actor URLs, patch infrastructure -- [`memory/feedback_patches.md`](memory/feedback_patches.md) — Patch authoring patterns, known fragile points, compose-path gotchas - -**Always read the relevant memory file before investigating or modifying anything in this repo.** +| File | When to read | +|---|---| +| [`memory/project_activitypub.md`](memory/project_activitypub.md) | Any AP / fediverse / reply threading work | +| [`memory/project_architecture.md`](memory/project_architecture.md) | Server layout, MongoDB, nginx, internal URLs | +| [`memory/feedback_patches.md`](memory/feedback_patches.md) | Writing or debugging patch scripts | --- ## Running ```sh -npm run serve # apply all patches then start Indiekit -npm run postinstall # re-apply patches after npm install (postinstall runs automatically) +npm run serve # preflights + all patches + start Indiekit +npm run postinstall # re-apply patches after npm install ``` -Do **not** restart with `node` directly — patches must run first. +Never start with `node` directly — patches must run first. --- ## Patch system -All fixes to `node_modules` live in `scripts/patch-*.mjs`. -Both `postinstall` and `serve` in `package.json` run them in order. +All node_modules fixes live in `scripts/patch-*.mjs`. Both `postinstall` and `serve` run them in order. -### Writing a patch +### Pattern ```js -const MARKER = "// [patch] my-patch-name"; // idempotency check -const OLD_SNIPPET = `exact source text`; -const NEW_SNIPPET = `replacement ${MARKER}`; +const MARKER = "// [patch] my-patch-name"; +const OLD_SNIPPET = `exact source text (spaces not tabs, exact line endings)`; +const NEW_SNIPPET = `replacement text ${MARKER}`; -// 1. Skip if MARKER already present +// 1. Read file — skip if MARKER already present // 2. Warn if OLD_SNIPPET not found (upstream changed) // 3. Replace + writeFile ``` -- Match node_modules text **byte-for-byte** (spaces, not tabs; exact line endings). -- Always include both candidate paths: `node_modules/@rmdes/...` and - `node_modules/@indiekit/indiekit/node_modules/@rmdes/...`. -- Template literals in patch strings: escape `` \` `` and `\${}`. -- Append new AP patches **after** `patch-ap-federation-bridge-base-url` in both - `postinstall` and `serve` in `package.json`. -- Some patches are `serve`-only (not `postinstall`) — check both when adding microsub patches. +### Rules + +- Include **both** candidate paths: `node_modules/@rmdes/...` and `node_modules/@indiekit/indiekit/node_modules/@rmdes/...` +- Escape template literals: `` \` `` and `\${}` +- Append new AP patches **after** `patch-ap-federation-bridge-base-url` in both `postinstall` and `serve` +- `patch-microsub-reader-ap-dispatch` is `serve`-only — check both scripts for microsub patches +- After writing a patch script, run it immediately (`node scripts/patch-*.mjs`) to verify it applies cleanly --- -## ActivityPub actor +## Architecture — things that affect code -| Field | Value | +### Two-jail setup + +``` +Internet → nginx (web jail 10.100.0.10) → Indiekit (node jail 10.100.0.20:3000) +``` + +The node jail **cannot reach its own public HTTPS URL**. Internal self-fetches must use `INTERNAL_FETCH_URL=http://10.100.0.20:3000` directly. All such fetches go through `_toInternalUrl()` (injected by `patch-micropub-fetch-internal-url`). + +### nginx / Fedify + +nginx must forward `Host: blog.giersig.eu` and `X-Forwarded-Proto: https` or AP lookups 302-redirect to the login page. See `patch-ap-federation-bridge-base-url`. + +`createFederation()` requires `allowPrivateAddress: true` (blog resolves to a LAN IP) and `signatureTimeWindow: { hours: 12 }` (Mastodon retries with old signatures). + +### MongoDB collections + +| Collection | Contents | |---|---| -| Handle | `svemagie` (env `AP_HANDLE`) | -| Full handle | `@svemagie@blog.giersig.eu` | -| Actor URL | `https://blog.giersig.eu/activitypub/users/svemagie` | -| Mastodon migration alias | `@svemagie@troet.cafe` (`AP_ALSO_KNOWN_AS`) | - -AP syndicator `info.uid = publicationUrl` (`https://blog.giersig.eu/`), `info.checked = true`. +| `posts` | Micropub post data — `properties.url` is the lookup key | +| `ap_timeline` | AP posts (inbound + outbound) — keyed by `uid` | +| `ap_notifications` | Mentions, replies, likes, boosts | +| `ap_followers` / `ap_following` | Actor URLs | +| `ap_activities` | Outbound/inbound activity log | +| `ap_profile` | Own actor (name, icon, url) | +| `ap_interactions` | Own likes/boosts | +| `ap_keys` | RSA + Ed25519 key pairs | +| `ap_featured` | Pinned posts | --- ## Post type discovery -`getPostType(postTypes, properties)` in `post-type-discovery.js`: +`getPostType(postTypes, properties)` checks **key presence only** — value doesn't matter: -| Property present | Type | -|---|---| -| `in-reply-to` | `reply` → `/replies/{slug}/` | -| `like-of` | `like` → `/likes/{slug}/` | -| `repost-of` | `repost` → `/reposts/{slug}/` | -| `photo` | `photo` → `/photos/{slug}/` | -| _(none)_ | `note` → `/notes/{slug}/` | +| Key present | Type | Saved at | +|---|---|---| +| `in-reply-to` | `reply` | `/replies/{slug}/` | +| `like-of` | `like` | `/likes/{slug}/` | +| `repost-of` | `repost` | `/reposts/{slug}/` | +| `photo` | `photo` | `/photos/{slug}/` | +| _(none)_ | `note` | `/notes/{slug}/` | -Only KEY presence matters, not value. If `in-reply-to` is silently missing from the JF2 -object, the post becomes `"note"` with no error — this is a common threading bug root cause. +If `in-reply-to` is silently absent, the post becomes a note **with no error**. This is the most common threading bug root cause. --- -## Reply threading — two compose paths +## Reply threading — compose paths -| Path | How AP target is pre-checked | +Three paths, different syndication mechanics: + +| Path | AP checkbox mechanism | |---|---| -| AP reader `/activitypub/admin/reader/compose` | `target.defaultChecked = target.checked === true` (patched) | -| Microsub reader `/microsub/admin/reader/compose` | `target.checked` from Micropub `q=config` (already `true` in config) | -| Mastodon client API `POST /api/v1/statuses` | `mp-syndicate-to` hardcoded to `publicationUrl` (always AP) | +| AP reader `/activitypub/admin/reader/compose` | `target.defaultChecked = target.checked === true` *(patched by `patch-ap-compose-default-checked`)* | +| Microsub reader `/microsub/admin/reader/compose` | `target.checked` from Micropub `q=config` — already `true` for AP syndicator | +| Mastodon client API `POST /api/v1/statuses` | `mp-syndicate-to` hardcoded to `publicationUrl` — always AP | -The AP reader template uses `target.defaultChecked`, NOT `target.checked` — these are different. -`patch-ap-compose-default-checked` maps the config's `checked: true` through to `defaultChecked`. +The AP reader template uses `target.defaultChecked`, **not** `target.checked`. These are different fields. -### Mastodon API reply-to-reply (patched) +### ap_timeline insertion timing -`POST /api/v1/statuses` resolves `in_reply_to_id` via `findTimelineItemById(ap_timeline, id)`. -Own posts were previously not inserted into `ap_timeline` until the Eleventy build webhook fired -(30–120 s). `patch-ap-mastodon-reply-threading` inserts a provisional item immediately after -`postContent.create()` using `addTimelineItem()` (`$setOnInsert` — idempotent). +Own posts reach `ap_timeline` via two paths: +- **Mastodon API**: inserted immediately after `postContent.create()` *(patched by `patch-ap-mastodon-reply-threading`)* +- **Micropub + syndication webhook**: inserted by syndicator after Eleventy build (30–120 s) + +Any new code path that creates posts should insert to `ap_timeline` immediately — otherwise `in_reply_to_id` lookups fail during the build window. + +### Status ID format + +`encodeCursor(published)` = ms-since-epoch string. `findTimelineItemById` resolves this with a ±1 s range query using MongoDB `$dateFromString` to handle TZ-offset ISO strings. --- -## ap_timeline +## ActivityPub syndicator -Key collection. Documents keyed by `uid`. +`syndicator.syndicate(properties)` does **not** filter by post type. A note and a reply both become `Create(Note)`. The difference is whether `inReplyTo` is set (from `properties["in-reply-to"]`). -- **Inbound posts** — inserted by `inbox-handlers.js` on Create/Update/Announce activities -- **Own posts** — inserted by `syndicator.js` after delivery (after build) AND now immediately - by `POST /api/v1/statuses` (via patch) -- **Status IDs** — `encodeCursor(published)` = ms-since-epoch string; `findTimelineItemById` - resolves them with a ±1 s MongoDB `$dateFromString` range query to handle TZ-offset strings +JF2 → AS2 mapping: + +| Post type | Activity | Notes | +|---|---|---| +| `note` / `reply` | `Create(Note)` | reply has `inReplyTo` | +| `like` | `Create(Note)` | bookmark framing (🔖 emoji) | +| `repost` | `Announce` | | +| `article` | `Create(Article)` | has `name` | + +Visibility: + +| Value | `to` | `cc` | +|---|---|---| +| `public` | `as:Public` | followers | +| `unlisted` | followers | `as:Public` | +| `followers` | followers | — | --- ## Fork dependencies -Four packages come from GitHub forks (not npm): +```sh +# Pull latest commit from a fork: +npm install github:svemagie/ +npm install github:svemagie/indiekit-endpoint-activitypub +``` | Package | Fork | |---|---| @@ -127,31 +163,37 @@ Four packages come from GitHub forks (not npm): | `@rmdes/indiekit-endpoint-microsub` | `github:svemagie/indiekit-endpoint-microsub#bookmarks-import` | | `@rmdes/indiekit-endpoint-youtube` | `github:svemagie/indiekit-endpoint-youtube` | -To pull latest fork commits: `npm install github:svemagie/`. +--- + +## Debugging — starting points + +| Symptom | First check | +|---|---| +| Reply created as "note" not "reply" | Is `in-reply-to` in the Micropub request? Check: form hidden field, `submitComposeController`, `findTimelineItemById` return value, `formEncodedToJf2` | +| Reply not federated to AP | Is `mp-syndicate-to` set? Check `target.defaultChecked` / `target.checked`, `getSyndicateToProperty` in `jf2.js` | +| AP lookup returns 302 / auth redirect | nginx not forwarding `Host`/`X-Forwarded-Proto` — see `patch-ap-federation-bridge-base-url` | +| `findTimelineItemById` returns null | Item not yet in `ap_timeline` (build not finished) or TZ-offset date mismatch — `$dateFromString` range query should catch offsets | +| Favourite/reblog hangs in Mastodon client | `resolveAuthor` timeout — `Promise.race` 5 s cap should prevent this | +| "Empty reply from server" on webmention poller | Poller routing through nginx (returns 444 for wrong Host) — must use `INDIEKIT_DIRECT_URL` | +| HTTP Signature verify errors flooding logs | Expected for deleted/migrated actors — suppressed to `fatal` level in federation-setup.js | +| "OAuth callback failed. Missing parameters." | `state` parameter not echoed — fixed in fork (`b54146c`) | +| AP object 410 / Tombstone | Post was deleted — correct, served by FEP-4f05 | --- -## Environment variables (key ones) +## Environment variables | Var | Purpose | |---|---| -| `AP_HANDLE` | ActivityPub handle (default: `svemagie` from `GITHUB_USERNAME`) | -| `PUBLICATION_URL` | Canonical blog URL (`https://blog.giersig.eu`) | -| `INDIEKIT_URL` | Application URL (same as publication URL here) | +| `AP_HANDLE` | AP handle (`svemagie`) | +| `AP_ALSO_KNOWN_AS` | Migration alias (`https://troet.cafe/users/svemagie`) | +| `AP_LOG_LEVEL` | Fedify log level (`info` default; `debug` for delivery tracing) | +| `AP_DEBUG` | `1` to enable Fedify debug dashboard at `/activitypub/__debug__/` | +| `PUBLICATION_URL` | Canonical blog URL | +| `INDIEKIT_URL` | Application URL (same as publication URL) | +| `INTERNAL_FETCH_URL` | Direct node jail URL for self-fetches (`http://10.100.0.20:3000`) | +| `INDIEKIT_BIND_HOST` | Jail IP for webmention poller direct connect | +| `REDIS_URL` | Redis for AP message queue + KV (production; without this, queue lost on restart) | | `MONGO_HOST` / `MONGO_URL` | MongoDB connection | -| `REDIS_URL` | Redis for AP message queue and KV store (production) | -| `AP_ALSO_KNOWN_AS` | Migration alias (old Mastodon handle) | | `GH_CONTENT_TOKEN` | GitHub token for writing posts to the `blog` repo | - ---- - -## Common debugging starting points - -- **Post created as "note" instead of "reply"** → `in-reply-to` missing from JF2. Check: form - hidden field, `submitComposeController`, Mastodon API `findTimelineItemById`, `formEncodedToJf2`. -- **Reply not federated to AP** → `mp-syndicate-to` not set. Check: `target.defaultChecked` / - `target.checked` in compose form, `getSyndicateToProperty` in `jf2.js`. -- **AP object lookup 302 redirect** → nginx not forwarding `Host` / `X-Forwarded-Proto`; see - `patch-ap-federation-bridge-base-url`. -- **Timeline item not found by Mastodon client** → `findTimelineItemById` date mismatch; stored - date is TZ-offset string, lookup decodes cursor to UTC ISO — relies on `$dateFromString` range. +| `SECRET` | JWT signing secret (webmention poller auth) |