From cd7f21e7fee7bff4ad77e26ac50ce7dc1c11396d Mon Sep 17 00:00:00 2001 From: Sven Date: Thu, 9 Apr 2026 16:05:05 +0200 Subject: [PATCH] docs: update memory for 2026-04-09 activitypub upgrade - project_activitypub: status IDs now MongoDB ObjectIds (not timestamp cursors); document new findTimelineItemById, resolveReplyIds, and updated patch behaviours for status-id, delete-fix, status-reply-id - feedback_patches: add lessons on chained-marker fragility, delta patches depending on deleted base patches, and pattern for handling upstream bug fixes gracefully Co-Authored-By: Claude Sonnet 4.6 --- memory/feedback_patches.md | 17 +++++++++ memory/project_activitypub.md | 72 ++++++++++++++++++----------------- 2 files changed, 54 insertions(+), 35 deletions(-) diff --git a/memory/feedback_patches.md b/memory/feedback_patches.md index 8bba0df0..cbd89ce1 100644 --- a/memory/feedback_patches.md +++ b/memory/feedback_patches.md @@ -34,6 +34,23 @@ const candidates = [ - **`patch-microsub-reader-ap-dispatch`** is in `serve` only (not `postinstall`). Reason unknown but may relate to timing or the microsub package being rebuilt differently. Check both scripts when adding new microsub patches. +- **Multi-part patches that chain off each other's MARKERs** — if Change B injects a MARKER that + Change C then targets, deleting Change B leaves Change C silently broken. Either keep the chain + intact, or rewrite the downstream change to match the upstream code directly (without the marker). +- **Delta-only patches that depend on a deleted base patch** — `patch-micropub-gitea-dispatch-conditional` + was written as a delta on a base patch that was later deleted. When a base patch is removed from + `package.json`, all delta patches that reference its output must be rewritten as standalone patches. + +## When Upstream Fixes a Patched Bug + +When an upstream upgrade fixes something your patch was correcting: +1. Add an "upstream fix indicator" string that appears in the newly-fixed upstream code. +2. In `applyPatch`, check for this indicator before trying to apply the OLD_SNIPPET. +3. Log `"already fixed upstream"` (not a warning) and skip gracefully. +4. Update the patch comment to note the date the upstream fix landed. + +This avoids noisy warnings on every `postinstall` while keeping the patch script as a fallback +for older upstream versions. ## AP Threading — Two Compose Paths diff --git a/memory/project_activitypub.md b/memory/project_activitypub.md index 5640af7b..4572f465 100644 --- a/memory/project_activitypub.md +++ b/memory/project_activitypub.md @@ -10,7 +10,9 @@ | `node_modules/@rmdes/indiekit-endpoint-activitypub/lib/jf2-to-as2.js` | Converts JF2 properties to ActivityPub Note/Create activity | | `node_modules/@rmdes/indiekit-endpoint-activitypub/lib/storage/timeline.js` | `addTimelineItem()` — atomic upsert (`$setOnInsert`) to `ap_timeline` | | `node_modules/@rmdes/indiekit-endpoint-activitypub/lib/mastodon/entities/status.js` | Serializes `ap_timeline` documents to Mastodon Status JSON | -| `node_modules/@rmdes/indiekit-endpoint-activitypub/lib/mastodon/helpers/pagination.js` | `encodeCursor(date)` / `decodeCursor(id)` — ms-since-epoch as status ID | +| `node_modules/@rmdes/indiekit-endpoint-activitypub/lib/mastodon/helpers/pagination.js` | `encodeCursor(date)` / `decodeCursor(id)` — ms-since-epoch cursors (for timeline pagination, NOT status IDs) | +| `node_modules/@rmdes/indiekit-endpoint-activitypub/lib/mastodon/helpers/resolve-reply-ids.js` | Batch-resolves `inReplyTo` URLs to MongoDB ObjectId strings via `ap_timeline` lookup | +| `node_modules/@rmdes/indiekit-endpoint-activitypub/lib/mastodon/helpers/id-mapping.js` | Maps remote actor URLs to stable local account IDs | | `node_modules/@indiekit/endpoint-micropub/lib/post-type-discovery.js` | `getPostType()` — returns "reply" if `in-reply-to` key present, else "note" | | `node_modules/@indiekit/endpoint-micropub/lib/post-data.js` | Micropub create pipeline: normalise → getPostType → save | | `node_modules/@rmdes/indiekit-endpoint-microsub/lib/controllers/reader.js` | Microsub reader compose: auto-selects syndication targets via `detectProtocol()` | @@ -38,13 +40,13 @@ User clicks reply on timeline item ``` Client sends POST /api/v1/statuses { status, in_reply_to_id } → findTimelineItemById(ap_timeline, in_reply_to_id) - → decodes cursor (ms-since-epoch) → looks up by published date + → collection.findOne({ _id: new ObjectId(id) }) ← ObjectId lookup since 2026-04-09 upgrade → inReplyTo = replyItem.uid || replyItem.url → jf2["in-reply-to"] = inReplyTo (if resolved) → jf2["mp-syndicate-to"] = [publicationUrl] (always set — AP-only) → postData.create → postContent.create - → addTimelineItem immediately [patch: ap-mastodon-reply-threading] - → returns minimal status JSON to client + → _tlItem = addTimelineItem(...) immediately [patch: ap-mastodon-reply-threading] + → returns status with id: _tlItem._id.toString() [patch: ap-mastodon-status-id] → build webhook → syndicator → AP delivery ``` @@ -54,9 +56,15 @@ The AP syndicator stores outbound posts with: - `uid = url = properties.url` (blog URL, e.g. `https://blog.giersig.eu/replies/slug/`) - `published = properties.published` (ISO 8601 with TZ offset, e.g. `"2026-03-21T16:33:50+01:00"`) - `inReplyTo = properties["in-reply-to"]` (original Mastodon URL or own blog URL) +- `_id` = MongoDB ObjectId (auto-generated by `addTimelineItem`) -The Mastodon API encodes status IDs as `encodeCursor(published)` = ms-since-epoch string. -`findTimelineItemById` uses a ±1 s range query with `$dateFromString` to handle TZ-offset strings. +**Status ID format (post 2026-04-09 upgrade):** MongoDB ObjectId string. +`findTimelineItemById` does `collection.findOne({ _id: new ObjectId(id) })`. +The `POST /api/v1/statuses` response now returns `_tlItem._id.toString()` as the status ID +(patched by `patch-ap-mastodon-status-id`) so subsequent `in_reply_to_id` lookups resolve correctly. + +`in_reply_to_id` in the status serializer is resolved via `resolveReplyIds()`, which batch-looks up +`ap_timeline` items by `uid`/`url` and returns `_id.toString()` — upstream now handles this properly. ## Syndication Target Config @@ -153,16 +161,15 @@ POST returns 401 → remote servers exhaust retries and stop delivering. `new URL(publicationUrl).host` (`"blog.giersig.eu"`) when `publicationUrl` is provided. **Effect:** HTTP Signature verification now succeeds for all inbound AP activities. -### `patch-ap-mastodon-status-id` *(2026-04-01)* +### `patch-ap-mastodon-status-id` *(updated 2026-04-09)* **File:** `lib/mastodon/routes/statuses.js` -**Problem:** `POST /api/v1/statuses` returned `id: String(Date.now())` — the wall-clock time of the -response. The `ap_timeline` item uses `published: data.properties.published`, set before the Gitea -write (which can take 5–15 s). When the client replies to the freshly created post, it sends -`in_reply_to_id: `, which is 5–15 s later than the stored `published` → the ±1 s -range query misses → `inReplyTo = null` → reply saved as note. -**Fix:** Use `encodeCursor(data.properties.published)` as the status ID in the creation response -(falls back to `String(Date.now())` if published is missing). Response ID now matches what -`findTimelineItemById` will resolve. +**Original problem (2026-04-01):** `POST /api/v1/statuses` returned `id: String(Date.now())`. +With timestamp-cursor IDs, this caused ±1 s range misses when the client immediately replied. +**Upstream change (2026-04-09):** `findTimelineItemById` switched to `new ObjectId(id)` lookup. +`String(Date.now())` is not a valid ObjectId → lookup always returns null → `inReplyTo = null`. +**Current fix:** Capture `addTimelineItem()` return value as `_tlItem`; use `_tlItem._id.toString()` +as the response ID (falls back to `String(Date.now())`). Response ID is now a valid ObjectId that +`findTimelineItemById` can resolve. ### `patch-ap-interactions-send-guard` *(2026-04-01)* **File:** `lib/mastodon/helpers/interactions.js` @@ -182,15 +189,15 @@ Root cause: the AP syndicator UID (`publicationUrl`) shares the same origin as t **Fix:** At the start of `syndicate()`, query `ap_activities` for an existing outbound Create/Announce/Update for `properties.url`. If found, return the existing URL without re-federating. -### `patch-ap-mastodon-delete-fix` *(2026-04-01)* +### `patch-ap-mastodon-delete-fix` *(updated 2026-04-09)* **File:** `lib/mastodon/routes/statuses.js` (delete route) + `index.js` -**Bug 1 (ReferenceError):** Delete route used `objectId` (undefined) instead of `item._id` from -`findTimelineItemById` → every delete threw ReferenceError → 500 → timeline entry never removed. -**Bug 2 (no AP broadcast):** Route called `postContent.delete()` directly, bypassing the Indiekit -syndicator framework → no `Delete(Note)` activity sent to followers → post persists on Mastodon. -**Fix:** (a) Add `broadcastDelete: (url) => pluginRef.broadcastDelete(url)` to -`mastodonPluginOptions` in `index.js`. (b) Call `req.app.locals.mastodonPluginOptions.broadcastDelete(postUrl)` -after removing the timeline entry. +**Bug 1 (ReferenceError) — now fixed upstream:** Delete route used `objectId` (undefined) instead +of `item._id`. Upstream (2026-04-09) fixed this natively; patch no longer applies Change B. +**Bug 2 (no AP broadcast) — still patched:** Route calls `postContent.delete()` directly, bypassing +the syndicator framework → no `Delete(Note)` activity sent to followers. +**Fix:** (a) Add `broadcastDelete: (url) => pluginRef.broadcastDelete(url)` to `mastodonPluginOptions` +in `index.js` (Change A — still applied). (b) Call `broadcastDelete(postUrl)` after the +`deleteOne` in the delete route (Change C — matches upstream code directly, no marker chain). ### `patch-micropub-delete-propagation` + `patch-bluesky-syndicator-delete` *(2026-04-01)* **Files:** `node_modules/@indiekit/endpoint-micropub/lib/action.js` + Bluesky syndicator @@ -211,19 +218,14 @@ non-followers not stored in `ap_timeline`. Also added else-if branch in `handleCreate` to store replies to own posts in `ap_timeline` even when sender is not in `ap_following`. -### `patch-ap-status-reply-id` *(2026-04-01)* +### `patch-ap-status-reply-id` *(updated 2026-04-09)* **Files:** `lib/mastodon/entities/status.js` + `lib/mastodon/routes/statuses.js` -**Problem:** `in_reply_to_id` in the status serializer was a tautological `item.inReplyTo ? null : null` -(unfilled TODO) — always `null`. Mastodon clients (Phanpy/Elk) use this field to display reply -threading; without it, own replies appear as standalone posts. -**Fix (two parts):** -(A) `status.js`: return `item.inReplyToId || null` instead of the tautological null. -(B) `statuses.js` POST handler: when pre-inserting own posts into `ap_timeline` (reply-threading -patch), also store `inReplyToId: inReplyToId || null` — the raw `in_reply_to_id` cursor from -the client is already a valid `encodeCursor` value. -**Note:** Inbound AP replies from remote servers still have `inReplyToId = null` (separate patch -needed). Own replies via the Mastodon client API are fully fixed. -**Effect:** Own replies are threaded correctly in Phanpy/Elk. +**Original problem (2026-04-01):** `in_reply_to_id` was tautological `item.inReplyTo ? null : null`. +**Change A — now fixed upstream (2026-04-09):** `status.js` now uses `replyIdMap?.get(item.inReplyTo)` +(resolved via `resolveReplyIds()` batch lookup). Patch detects the upstream fix and skips Change A silently. +**Change B — still applied:** `statuses.js` POST handler pre-insert stores +`inReplyToId: inReplyToId || null` so the item is available for `resolveReplyIds` lookups. +**Effect:** Own replies are threaded correctly in Phanpy/Elk; inbound AP replies use `resolveReplyIds`. ---