docs: update memory for 2026-04-09 activitypub upgrade
Deploy Indiekit Server / deploy (push) Successful in 1m18s

- project_activitypub: status IDs now MongoDB ObjectIds (not timestamp
  cursors); document new findTimelineItemById, resolveReplyIds, and
  updated patch behaviours for status-id, delete-fix, status-reply-id
- feedback_patches: add lessons on chained-marker fragility, delta
  patches depending on deleted base patches, and pattern for handling
  upstream bug fixes gracefully

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Sven
2026-04-09 16:05:05 +02:00
parent 5ac8f1f43b
commit cd7f21e7fe
2 changed files with 54 additions and 35 deletions
+17
View File
@@ -34,6 +34,23 @@ const candidates = [
- **`patch-microsub-reader-ap-dispatch`** is in `serve` only (not `postinstall`). Reason unknown - **`patch-microsub-reader-ap-dispatch`** is in `serve` only (not `postinstall`). Reason unknown
but may relate to timing or the microsub package being rebuilt differently. Check both scripts but may relate to timing or the microsub package being rebuilt differently. Check both scripts
when adding new microsub patches. when adding new microsub patches.
- **Multi-part patches that chain off each other's MARKERs** — if Change B injects a MARKER that
Change C then targets, deleting Change B leaves Change C silently broken. Either keep the chain
intact, or rewrite the downstream change to match the upstream code directly (without the marker).
- **Delta-only patches that depend on a deleted base patch** — `patch-micropub-gitea-dispatch-conditional`
was written as a delta on a base patch that was later deleted. When a base patch is removed from
`package.json`, all delta patches that reference its output must be rewritten as standalone patches.
## When Upstream Fixes a Patched Bug
When an upstream upgrade fixes something your patch was correcting:
1. Add an "upstream fix indicator" string that appears in the newly-fixed upstream code.
2. In `applyPatch`, check for this indicator before trying to apply the OLD_SNIPPET.
3. Log `"already fixed upstream"` (not a warning) and skip gracefully.
4. Update the patch comment to note the date the upstream fix landed.
This avoids noisy warnings on every `postinstall` while keeping the patch script as a fallback
for older upstream versions.
## AP Threading — Two Compose Paths ## AP Threading — Two Compose Paths
+37 -35
View File
@@ -10,7 +10,9 @@
| `node_modules/@rmdes/indiekit-endpoint-activitypub/lib/jf2-to-as2.js` | Converts JF2 properties to ActivityPub Note/Create activity | | `node_modules/@rmdes/indiekit-endpoint-activitypub/lib/jf2-to-as2.js` | Converts JF2 properties to ActivityPub Note/Create activity |
| `node_modules/@rmdes/indiekit-endpoint-activitypub/lib/storage/timeline.js` | `addTimelineItem()` — atomic upsert (`$setOnInsert`) to `ap_timeline` | | `node_modules/@rmdes/indiekit-endpoint-activitypub/lib/storage/timeline.js` | `addTimelineItem()` — atomic upsert (`$setOnInsert`) to `ap_timeline` |
| `node_modules/@rmdes/indiekit-endpoint-activitypub/lib/mastodon/entities/status.js` | Serializes `ap_timeline` documents to Mastodon Status JSON | | `node_modules/@rmdes/indiekit-endpoint-activitypub/lib/mastodon/entities/status.js` | Serializes `ap_timeline` documents to Mastodon Status JSON |
| `node_modules/@rmdes/indiekit-endpoint-activitypub/lib/mastodon/helpers/pagination.js` | `encodeCursor(date)` / `decodeCursor(id)` — ms-since-epoch as status ID | | `node_modules/@rmdes/indiekit-endpoint-activitypub/lib/mastodon/helpers/pagination.js` | `encodeCursor(date)` / `decodeCursor(id)` — ms-since-epoch cursors (for timeline pagination, NOT status IDs) |
| `node_modules/@rmdes/indiekit-endpoint-activitypub/lib/mastodon/helpers/resolve-reply-ids.js` | Batch-resolves `inReplyTo` URLs to MongoDB ObjectId strings via `ap_timeline` lookup |
| `node_modules/@rmdes/indiekit-endpoint-activitypub/lib/mastodon/helpers/id-mapping.js` | Maps remote actor URLs to stable local account IDs |
| `node_modules/@indiekit/endpoint-micropub/lib/post-type-discovery.js` | `getPostType()` — returns "reply" if `in-reply-to` key present, else "note" | | `node_modules/@indiekit/endpoint-micropub/lib/post-type-discovery.js` | `getPostType()` — returns "reply" if `in-reply-to` key present, else "note" |
| `node_modules/@indiekit/endpoint-micropub/lib/post-data.js` | Micropub create pipeline: normalise → getPostType → save | | `node_modules/@indiekit/endpoint-micropub/lib/post-data.js` | Micropub create pipeline: normalise → getPostType → save |
| `node_modules/@rmdes/indiekit-endpoint-microsub/lib/controllers/reader.js` | Microsub reader compose: auto-selects syndication targets via `detectProtocol()` | | `node_modules/@rmdes/indiekit-endpoint-microsub/lib/controllers/reader.js` | Microsub reader compose: auto-selects syndication targets via `detectProtocol()` |
@@ -38,13 +40,13 @@ User clicks reply on timeline item
``` ```
Client sends POST /api/v1/statuses { status, in_reply_to_id } Client sends POST /api/v1/statuses { status, in_reply_to_id }
→ findTimelineItemById(ap_timeline, in_reply_to_id) → findTimelineItemById(ap_timeline, in_reply_to_id)
decodes cursor (ms-since-epoch) → looks up by published date collection.findOne({ _id: new ObjectId(id) }) ← ObjectId lookup since 2026-04-09 upgrade
→ inReplyTo = replyItem.uid || replyItem.url → inReplyTo = replyItem.uid || replyItem.url
→ jf2["in-reply-to"] = inReplyTo (if resolved) → jf2["in-reply-to"] = inReplyTo (if resolved)
→ jf2["mp-syndicate-to"] = [publicationUrl] (always set — AP-only) → jf2["mp-syndicate-to"] = [publicationUrl] (always set — AP-only)
→ postData.create → postContent.create → postData.create → postContent.create
→ addTimelineItem immediately [patch: ap-mastodon-reply-threading] _tlItem = addTimelineItem(...) immediately [patch: ap-mastodon-reply-threading]
→ returns minimal status JSON to client → returns status with id: _tlItem._id.toString() [patch: ap-mastodon-status-id]
→ build webhook → syndicator → AP delivery → build webhook → syndicator → AP delivery
``` ```
@@ -54,9 +56,15 @@ The AP syndicator stores outbound posts with:
- `uid = url = properties.url` (blog URL, e.g. `https://blog.giersig.eu/replies/slug/`) - `uid = url = properties.url` (blog URL, e.g. `https://blog.giersig.eu/replies/slug/`)
- `published = properties.published` (ISO 8601 with TZ offset, e.g. `"2026-03-21T16:33:50+01:00"`) - `published = properties.published` (ISO 8601 with TZ offset, e.g. `"2026-03-21T16:33:50+01:00"`)
- `inReplyTo = properties["in-reply-to"]` (original Mastodon URL or own blog URL) - `inReplyTo = properties["in-reply-to"]` (original Mastodon URL or own blog URL)
- `_id` = MongoDB ObjectId (auto-generated by `addTimelineItem`)
The Mastodon API encodes status IDs as `encodeCursor(published)` = ms-since-epoch string. **Status ID format (post 2026-04-09 upgrade):** MongoDB ObjectId string.
`findTimelineItemById` uses a ±1 s range query with `$dateFromString` to handle TZ-offset strings. `findTimelineItemById` does `collection.findOne({ _id: new ObjectId(id) })`.
The `POST /api/v1/statuses` response now returns `_tlItem._id.toString()` as the status ID
(patched by `patch-ap-mastodon-status-id`) so subsequent `in_reply_to_id` lookups resolve correctly.
`in_reply_to_id` in the status serializer is resolved via `resolveReplyIds()`, which batch-looks up
`ap_timeline` items by `uid`/`url` and returns `_id.toString()` — upstream now handles this properly.
## Syndication Target Config ## Syndication Target Config
@@ -153,16 +161,15 @@ POST returns 401 → remote servers exhaust retries and stop delivering.
`new URL(publicationUrl).host` (`"blog.giersig.eu"`) when `publicationUrl` is provided. `new URL(publicationUrl).host` (`"blog.giersig.eu"`) when `publicationUrl` is provided.
**Effect:** HTTP Signature verification now succeeds for all inbound AP activities. **Effect:** HTTP Signature verification now succeeds for all inbound AP activities.
### `patch-ap-mastodon-status-id` *(2026-04-01)* ### `patch-ap-mastodon-status-id` *(updated 2026-04-09)*
**File:** `lib/mastodon/routes/statuses.js` **File:** `lib/mastodon/routes/statuses.js`
**Problem:** `POST /api/v1/statuses` returned `id: String(Date.now())` — the wall-clock time of the **Original problem (2026-04-01):** `POST /api/v1/statuses` returned `id: String(Date.now())`.
response. The `ap_timeline` item uses `published: data.properties.published`, set before the Gitea With timestamp-cursor IDs, this caused ±1 s range misses when the client immediately replied.
write (which can take 515 s). When the client replies to the freshly created post, it sends **Upstream change (2026-04-09):** `findTimelineItemById` switched to `new ObjectId(id)` lookup.
`in_reply_to_id: <Date.now() id>`, which is 515 s later than the stored `published` → the ±1 s `String(Date.now())` is not a valid ObjectId → lookup always returns null → `inReplyTo = null`.
range query misses → `inReplyTo = null` → reply saved as note. **Current fix:** Capture `addTimelineItem()` return value as `_tlItem`; use `_tlItem._id.toString()`
**Fix:** Use `encodeCursor(data.properties.published)` as the status ID in the creation response as the response ID (falls back to `String(Date.now())`). Response ID is now a valid ObjectId that
(falls back to `String(Date.now())` if published is missing). Response ID now matches what `findTimelineItemById` can resolve.
`findTimelineItemById` will resolve.
### `patch-ap-interactions-send-guard` *(2026-04-01)* ### `patch-ap-interactions-send-guard` *(2026-04-01)*
**File:** `lib/mastodon/helpers/interactions.js` **File:** `lib/mastodon/helpers/interactions.js`
@@ -182,15 +189,15 @@ Root cause: the AP syndicator UID (`publicationUrl`) shares the same origin as t
**Fix:** At the start of `syndicate()`, query `ap_activities` for an existing outbound **Fix:** At the start of `syndicate()`, query `ap_activities` for an existing outbound
Create/Announce/Update for `properties.url`. If found, return the existing URL without re-federating. Create/Announce/Update for `properties.url`. If found, return the existing URL without re-federating.
### `patch-ap-mastodon-delete-fix` *(2026-04-01)* ### `patch-ap-mastodon-delete-fix` *(updated 2026-04-09)*
**File:** `lib/mastodon/routes/statuses.js` (delete route) + `index.js` **File:** `lib/mastodon/routes/statuses.js` (delete route) + `index.js`
**Bug 1 (ReferenceError):** Delete route used `objectId` (undefined) instead of `item._id` from **Bug 1 (ReferenceError) — now fixed upstream:** Delete route used `objectId` (undefined) instead
`findTimelineItemById` → every delete threw ReferenceError → 500 → timeline entry never removed. of `item._id`. Upstream (2026-04-09) fixed this natively; patch no longer applies Change B.
**Bug 2 (no AP broadcast):** Route called `postContent.delete()` directly, bypassing the Indiekit **Bug 2 (no AP broadcast) — still patched:** Route calls `postContent.delete()` directly, bypassing
syndicator framework → no `Delete(Note)` activity sent to followers → post persists on Mastodon. the syndicator framework → no `Delete(Note)` activity sent to followers.
**Fix:** (a) Add `broadcastDelete: (url) => pluginRef.broadcastDelete(url)` to **Fix:** (a) Add `broadcastDelete: (url) => pluginRef.broadcastDelete(url)` to `mastodonPluginOptions`
`mastodonPluginOptions` in `index.js`. (b) Call `req.app.locals.mastodonPluginOptions.broadcastDelete(postUrl)` in `index.js` (Change A — still applied). (b) Call `broadcastDelete(postUrl)` after the
after removing the timeline entry. `deleteOne` in the delete route (Change C — matches upstream code directly, no marker chain).
### `patch-micropub-delete-propagation` + `patch-bluesky-syndicator-delete` *(2026-04-01)* ### `patch-micropub-delete-propagation` + `patch-bluesky-syndicator-delete` *(2026-04-01)*
**Files:** `node_modules/@indiekit/endpoint-micropub/lib/action.js` + Bluesky syndicator **Files:** `node_modules/@indiekit/endpoint-micropub/lib/action.js` + Bluesky syndicator
@@ -211,19 +218,14 @@ non-followers not stored in `ap_timeline`.
Also added else-if branch in `handleCreate` to store replies to own posts in `ap_timeline` even Also added else-if branch in `handleCreate` to store replies to own posts in `ap_timeline` even
when sender is not in `ap_following`. when sender is not in `ap_following`.
### `patch-ap-status-reply-id` *(2026-04-01)* ### `patch-ap-status-reply-id` *(updated 2026-04-09)*
**Files:** `lib/mastodon/entities/status.js` + `lib/mastodon/routes/statuses.js` **Files:** `lib/mastodon/entities/status.js` + `lib/mastodon/routes/statuses.js`
**Problem:** `in_reply_to_id` in the status serializer was a tautological `item.inReplyTo ? null : null` **Original problem (2026-04-01):** `in_reply_to_id` was tautological `item.inReplyTo ? null : null`.
(unfilled TODO) — always `null`. Mastodon clients (Phanpy/Elk) use this field to display reply **Change A — now fixed upstream (2026-04-09):** `status.js` now uses `replyIdMap?.get(item.inReplyTo)`
threading; without it, own replies appear as standalone posts. (resolved via `resolveReplyIds()` batch lookup). Patch detects the upstream fix and skips Change A silently.
**Fix (two parts):** **Change B — still applied:** `statuses.js` POST handler pre-insert stores
(A) `status.js`: return `item.inReplyToId || null` instead of the tautological null. `inReplyToId: inReplyToId || null` so the item is available for `resolveReplyIds` lookups.
(B) `statuses.js` POST handler: when pre-inserting own posts into `ap_timeline` (reply-threading **Effect:** Own replies are threaded correctly in Phanpy/Elk; inbound AP replies use `resolveReplyIds`.
patch), also store `inReplyToId: inReplyToId || null` — the raw `in_reply_to_id` cursor from
the client is already a valid `encodeCursor` value.
**Note:** Inbound AP replies from remote servers still have `inReplyToId = null` (separate patch
needed). Own replies via the Mastodon client API are fully fixed.
**Effect:** Own replies are threaded correctly in Phanpy/Elk.
--- ---