From 981ae7c885a48e56fc73a0a26bab69b3409d8fa1 Mon Sep 17 00:00:00 2001 From: Sven Date: Thu, 9 Apr 2026 16:07:39 +0200 Subject: [PATCH] docs: update README for 2026-04-09 activitypub fork upgrade - Lockfile caveat: bump fork HEAD to c8ca991, describe ObjectId ID migration, resolveReplyIds, upstream bug fixes, new helpers - Patch descriptions: update status-id (ObjectId approach), status-reply-id (Change A upstream fix detection), delete-fix (Change B upstream fix) - Changelog: add 2026-04-09 entry; annotate superseded 2026-04-01 entries Co-Authored-By: Claude Sonnet 4.6 --- README.md | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 5cd4945a..17225fe2 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ Four packages are installed directly from Gitea forks rather than the npm regist In `package.json` these use the `git+https://gitea.giersig.eu/svemagie/repo` syntax so npm fetches them directly from Gitea on install. -> **Lockfile caveat:** The fork dependency is resolved to a specific commit in `package-lock.json`. When fixes are pushed to the fork, run `npm install git+https://gitea.giersig.eu/svemagie/indiekit-endpoint-activitypub` to pull the latest commit. The fork HEAD is at `b54146c` (upstream v3.9.x merged: Fedify 2.1.0, 5 FEPs — Tombstone/soft-delete, Activity Intents, indexable actor, NodeInfo enrichment, Collection Sync; security audit — XSS/CSRF/OAuth scope enforcement, rate limiting, token expiry, secret hashing; architecture refactor — syndicator.js, batch-broadcast.js, init-indexes.js, CSS split into 15 files; plus all fork patches: DM support, pin/unpin status, edit post, favourite/reblog timeout guard, raw signed fetch fallback, timezone-aware status lookup, own Micropub posts mirrored into ap_timeline, inbox HTTP Signature noise suppressed, OAuth `state` parameter echo fix). +> **Lockfile caveat:** The fork dependency is resolved to a specific commit in `package-lock.json`. When fixes are pushed to the fork, run `npm install git+https://gitea.giersig.eu/svemagie/indiekit-endpoint-activitypub` to pull the latest commit. The fork HEAD is at `c8ca991` (2026-04-09: status IDs migrated from timestamp cursors to MongoDB ObjectIds — `findTimelineItemById` now does `findOne({ _id: new ObjectId(id) })`; `in_reply_to_id` serializer uses `resolveReplyIds()` batch lookup; upstream fixed `objectId` ReferenceError in DELETE route; draft/unlisted syndication guards moved into `index.js` natively; new helpers: `resolve-reply-ids.js`, `id-mapping.js`; builds on `b54146c`: upstream v3.9.x merged — Fedify 2.1.0, 5 FEPs, security/perf audit, architecture refactor; plus all fork patches: DM support, pin/unpin, edit post, favourite/reblog timeout guard, raw signed fetch fallback, timezone-aware status lookup, own Micropub posts mirrored into ap_timeline, inbox HTTP Signature noise suppressed, OAuth `state` parameter echo fix). --- @@ -423,10 +423,10 @@ In the AP reader compose form (`/activitypub/admin/reader/compose`), the upstrea After `POST /api/v1/statuses` (Phanpy/Elk creates a post), the handler intentionally did not insert the new post into `ap_timeline` — it relied on the Eleventy build webhook firing 30–120 s later. If the user replies to their own new post during that window, `findTimelineItemById` returns null, `in_reply_to_id` is silently dropped, and the follow-up reply is classified as a "note" (wrong post type, wrong URL path, no `inReplyTo` in AP output, broken thread on Mastodon). Fix: immediately after `postContent.create()`, inserts a provisional timeline item via `addTimelineItem()` using `$setOnInsert` (idempotent — syndicator's later upsert is a no-op). **`patch-ap-mastodon-status-id.mjs`** -`POST /api/v1/statuses` previously returned `id: String(Date.now())` — the wall-clock time of the HTTP response. But the `ap_timeline` item is stored with `published: data.properties.published`, which is set before the Gitea write (5–15 s earlier). When a client immediately replies to the freshly created post, it sends `in_reply_to_id` equal to `Date.now()`, which is 5–15 s later than the stored `published`. The ±1 s range query in `findTimelineItemById` misses, so `inReplyTo = null` and the reply is saved as a note. Fix: use `encodeCursor(data.properties.published)` as the status ID in the creation response (falls back to `String(Date.now())` if `published` is missing). The returned ID now matches what `findTimelineItemById` will resolve. +`POST /api/v1/statuses` returns `id: String(Date.now())` by default — not a valid MongoDB ObjectId. Since the 2026-04-09 upstream upgrade, `findTimelineItemById` looks up items with `findOne({ _id: new ObjectId(id) })`. A `Date.now()` string fails ObjectId parsing → lookup returns null → `inReplyTo = null` → reply saved as note. Fix: capture the `addTimelineItem()` return value as `_tlItem` and use `_tlItem._id.toString()` as the creation response ID (falls back to `String(Date.now())`). The returned ID is now a valid ObjectId that `findTimelineItemById` can resolve. *(Originally fixed a timestamp-cursor mismatch; re-patched after upstream switched to ObjectId IDs.)* **`patch-ap-status-reply-id.mjs`** -Two-part fix for `in_reply_to_id` always being `null` in the Mastodon status serializer. (A) `status.js`: the field was a tautological `item.inReplyTo ? null : null` (unfilled TODO) — changed to `item.inReplyToId || null`. (B) `statuses.js` POST handler: when pre-inserting own posts into `ap_timeline` (reply-threading patch), also stores `inReplyToId: inReplyToId || null` — the raw `in_reply_to_id` cursor from the client is already a valid `encodeCursor` value. Inbound AP replies from remote servers will still have `inReplyToId = null` until a separate patch populates it from `ap_timeline` lookups. +Two-part fix for `in_reply_to_id` always being `null` in the Mastodon status serializer. (A) `status.js`: the field was a tautological `item.inReplyTo ? null : null` (unfilled TODO). *Upstream (2026-04-09) fixed this properly via `resolveReplyIds()` batch lookup — patch detects the upstream fix and skips Change A silently.* (B) `statuses.js` POST handler: when pre-inserting own posts into `ap_timeline` (reply-threading patch), also stores `inReplyToId: inReplyToId || null` so the item is reachable by `resolveReplyIds`. **`patch-ap-interactions-send-guard.mjs`** `likePost` and `boostPost` in `lib/mastodon/helpers/interactions.js` called `ctx.sendActivity()` without try/catch. Any Fedify or Redis error propagated to the caller → 500 response → the `ap_interactions` DB write never ran → interaction not recorded locally. Fix: wrap both `sendActivity` calls in try/catch so delivery failures are non-fatal. Interaction is still recorded in `ap_interactions`; client sees correct UI state. @@ -435,7 +435,7 @@ Two-part fix for `in_reply_to_id` always being `null` in the Mastodon status ser The CI webhook calls `/syndicate?source_url=X&force=true` after every Eleventy build. When `syndicateToTargets()` saves the syndication URL it commits to Gitea → triggers another build → second CI call also hits the syndicate endpoint → duplicate `Create(Note)` activity sent. Root cause: the AP syndicator UID (`publicationUrl`) shares the same origin as the syndication URL (`properties.url`), so `force` mode re-selects it. Fix: at the start of `syndicate()`, queries `ap_activities` for an existing outbound Create/Announce/Update for `properties.url`. If found, returns the existing URL without re-federating. **`patch-ap-mastodon-delete-fix.mjs`** -Two bugs in the Mastodon API delete route. Bug 1 (ReferenceError): the route used `objectId` (undefined) instead of `item._id` from `findTimelineItemById` → every delete threw ReferenceError → 500 → timeline entry never removed. Bug 2 (no AP broadcast): the route called `postContent.delete()` directly, bypassing the Indiekit syndicator framework → no `Delete(Note)` activity sent to followers → post persisted on Mastodon. Fix: (a) adds `broadcastDelete: (url) => pluginRef.broadcastDelete(url)` to `mastodonPluginOptions` in `index.js`; (b) calls `broadcastDelete(postUrl)` after removing the timeline entry. +Two bugs in the Mastodon API delete route. Bug 1 (ReferenceError): the route used `objectId` (undefined) instead of `item._id` → every delete threw ReferenceError → 500 → timeline entry never removed. *Upstream (2026-04-09) fixed this natively; patch no longer applies this change.* Bug 2 (no AP broadcast — still patched): the route calls `postContent.delete()` directly, bypassing the syndicator framework → no `Delete(Note)` sent to followers. Fix: (a) adds `broadcastDelete: (url) => pluginRef.broadcastDelete(url)` to `mastodonPluginOptions` in `index.js`; (b) calls `broadcastDelete(postUrl)` after `deleteOne` in the delete route. **`patch-ap-inbox-publication-url.mjs`** `collections._publicationUrl` was never set in `federation-setup.js`, so every `pubUrl && objectId.startsWith(pubUrl)` guard in `handleCreate`/`handleAnnounce` always evaluated to `undefined`. Result: no reply notifications, no boost notifications for own content, replies from non-followers not stored in `ap_timeline`. Fix: sets `collections._publicationUrl = publicationUrl` before `registerInboxListeners()`. Also adds an else-if branch in `handleCreate` to store replies to own posts in `ap_timeline` even when the sender is not in `ap_following`. @@ -876,6 +876,18 @@ A CommonJS preload module that runs a Prometheus scrape endpoint on port 9209 (c ## Changelog +### 2026-04-09 + +**chore(deps): update indiekit-endpoint-activitypub fork** (`c8ca991`) +Pulled latest commit from svemagie/indiekit-endpoint-activitypub. Key upstream changes: status IDs migrated from timestamp cursors to MongoDB ObjectIds (`findTimelineItemById` now does `findOne({ _id: new ObjectId(id) })`); `in_reply_to_id` serialiser uses `resolveReplyIds()` batch lookup (`resolve-reply-ids.js`) instead of the tautological null; `objectId` ReferenceError in DELETE route fixed upstream; draft/unlisted syndication guards moved from patches into `index.js` natively. + +**fix(patches): adapt AP patches for ObjectId status ID migration** +Four patches updated to match the new upstream: +- `patch-ap-mastodon-status-id`: rewritten — captures `addTimelineItem()` return value, uses `_tlItem._id.toString()` as the `POST /api/v1/statuses` response ID so `findTimelineItemById` (now ObjectId-based) resolves correctly for immediate reply threading. +- `patch-ap-mastodon-delete-fix`: Change B (objectId → item._id) dropped (upstream fixed); Change C (broadcastDelete call) updated to match upstream code directly without chaining off Change B's marker. +- `patch-ap-status-reply-id`: Change A (tautological null) now detects the upstream `resolveReplyIds` fix and skips silently; Change B (store `inReplyToId` in timeline insert) still applied. +- `patch-micropub-gitea-dispatch-conditional`: rewritten as standalone patch — injects `_dispatchGiteaBuild` helper + conditional guard in one step, no longer requires the deleted base dispatch patch. + ### 2026-04-01 **chore(deps): upgrade @fedify/* 2.1.1 → 2.1.3** (`65c1813`) @@ -894,10 +906,10 @@ nginx forwards the upstream IP as the `Host` header. Fedify includes `host` in t `patch-ap-inbox-publication-url`: sets `collections._publicationUrl` in `federation-setup.js` before inbox listener registration, enabling `handleCreate`/`handleAnnounce`/`handleLike` notifications for own content. Also stores replies to own posts from non-followers in `ap_timeline`. **fix(ap): status creation response id mismatch breaks immediate reply threading** (`patch-ap-mastodon-status-id`) -`POST /api/v1/statuses` returned `id: String(Date.now())` (response time), but `ap_timeline` stores `published` (set pre-Gitea-write, 5–15 s earlier). Immediate follow-up replies sent `in_reply_to_id = Date.now()`, missing the ±1 s range query. Fixed: use `encodeCursor(data.properties.published)` as the response ID. +`POST /api/v1/statuses` returned `id: String(Date.now())` (response time), but `ap_timeline` stores `published` (set pre-Gitea-write, 5–15 s earlier). Immediate follow-up replies sent `in_reply_to_id = Date.now()`, missing the ±1 s range query. Fixed: use `encodeCursor(data.properties.published)` as the response ID. *(Superseded 2026-04-09: upstream switched to ObjectId IDs; patch rewritten to use `_tlItem._id.toString()`.)* **fix(ap): in_reply_to_id always null in Mastodon status serializer** (`patch-ap-status-reply-id`) -Tautological `item.inReplyTo ? null : null` in `status.js` fixed to `item.inReplyToId || null`. POST handler now also stores `inReplyToId` in the timeline item so own replies are threaded in Phanpy/Elk. +Tautological `item.inReplyTo ? null : null` in `status.js` fixed to `item.inReplyToId || null`. POST handler now also stores `inReplyToId` in the timeline item so own replies are threaded in Phanpy/Elk. *(Change A superseded 2026-04-09: upstream adopted `resolveReplyIds()` batch lookup; patch now skips Change A silently.)* **fix(ap): favourite/reblog error crashes interaction recording** (`patch-ap-interactions-send-guard`) `sendActivity` in `likePost`/`boostPost` lacked try/catch; any Redis/Fedify error caused a 500 and skipped the `ap_interactions` DB write. Wrapped in try/catch so delivery failures are non-fatal. @@ -906,7 +918,7 @@ Tautological `item.inReplyTo ? null : null` in `status.js` fixed to `item.inRepl CI webhook triggers twice per post (Gitea commit from syndication URL save triggers another build). `patch-ap-syndicate-dedup` checks `ap_activities` for an existing outbound activity for the URL and short-circuits if found. **fix(ap): Mastodon API delete: ReferenceError + no AP broadcast** (`patch-ap-mastodon-delete-fix`) -Delete route used undefined `objectId` (should be `item._id`) → ReferenceError → 500. Also called `postContent.delete()` directly → no `Delete(Note)` sent to followers. Fixed: corrected variable + wired `broadcastDelete` through `mastodonPluginOptions`. +Delete route used undefined `objectId` (should be `item._id`) → ReferenceError → 500. Also called `postContent.delete()` directly → no `Delete(Note)` sent to followers. Fixed: corrected variable + wired `broadcastDelete` through `mastodonPluginOptions`. *(ReferenceError fix superseded 2026-04-09: upstream fixed natively; broadcastDelete wiring still patched.)* ### 2026-03-27