From 0c84913ac73eda0a2daaf7211f5c33de189ae3f9 Mon Sep 17 00:00:00 2001 From: Ricardo Date: Mon, 16 Mar 2026 15:23:56 +0100 Subject: [PATCH] chore: move docs/plans/audits to centralized documentation-central MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Plans → documentation-central/plans/ Audit → documentation-central/audits/ Research → documentation-central/docs/ Confab-Link: http://localhost:8080/sessions/d6567f44-c576-4acd-9c8c-454aa58fbde9 --- docs/activitypub-coverage-audit.md | 403 ------- .../2026-02-28-explore-tabbed-redesign.md | 513 --------- ...2026-03-02-reader-timeline-enhancements.md | 40 - .../2026-03-03-reader-improvements-plan.md | 996 ------------------ ...2026-03-13-activitypub-high-impact-gaps.md | 826 --------------- .../2026-03-03-elk-phanpy-comparison.md | 440 -------- 6 files changed, 3218 deletions(-) delete mode 100644 docs/activitypub-coverage-audit.md delete mode 100644 docs/plans/2026-02-28-explore-tabbed-redesign.md delete mode 100644 docs/plans/2026-03-02-reader-timeline-enhancements.md delete mode 100644 docs/plans/2026-03-03-reader-improvements-plan.md delete mode 100644 docs/plans/2026-03-13-activitypub-high-impact-gaps.md delete mode 100644 docs/research/2026-03-03-elk-phanpy-comparison.md diff --git a/docs/activitypub-coverage-audit.md b/docs/activitypub-coverage-audit.md deleted file mode 100644 index b08da52..0000000 --- a/docs/activitypub-coverage-audit.md +++ /dev/null @@ -1,403 +0,0 @@ -# ActivityPub Coverage Audit: @rmdes/indiekit-endpoint-activitypub vs Fedify 2.0 - -**Date:** 2026-03-13 -**Plugin Version:** 2.9.2 -**Fedify Version:** 2.0.0 -**Auditor:** Claude Code (Opus 4.6) - ---- - -## Legend - -- **Implemented** — fully working in production -- **Partial** — some aspects implemented, gaps remain -- **Not implemented** — Fedify supports it, we don't use it - ---- - -## 1. Inbound Activity Handlers - -All handlers are in `lib/inbox-listeners.js`. Fedify dispatches inbound activities to registered listeners via `setInboxListeners()`. - -| Activity Type | Fedify Support | Our Implementation | Status | -|---|---|---|---| -| `Follow` | Full | Auto-accept, store follower in `ap_followers`, create notification, log to `ap_activities` (lines 90–149) | **Implemented** | -| `Accept` | Full | Updates `ap_following` entry to `source: "federation"`, clears retry fields (lines 194–235) | **Implemented** | -| `Reject` | Full | Marks `ap_following` entry as `source: "rejected"` (lines 236–267) | **Implemented** | -| `Undo(Follow)` | Full | Removes from `ap_followers` (lines 151–170) | **Implemented** | -| `Undo(Like)` | Full | Removes from `ap_activities` (lines 171–183) | **Implemented** | -| `Undo(Announce)` | Full | Removes from `ap_activities` (lines 184–193) | **Implemented** | -| `Like` | Full | Filtered to our content only (`objectId.startsWith(publicationUrl)`), stores notification + activity log (lines 268–317) | **Implemented** | -| `Announce` | Full | Dual path: boosts of our posts → notification; boosts from followed accounts → `ap_timeline` with quote enrichment (lines 318–412) | **Implemented** | -| `Create` | Full | Four paths: DMs → `ap_messages`; replies to us → notification; mentions → notification; followed accounts → `ap_timeline` with link preview + quote enrichment (lines 413–639) | **Implemented** | -| `Delete` | Full | Removes from `ap_activities` + `ap_timeline` (lines 640–649) | **Implemented** | -| `Update` | Full | Post updates → `ap_timeline` content refresh; profile updates → follower data refresh (lines 672–735) | **Implemented** | -| `Move` | Full | Updates follower `actorUrl` to new address, stores `movedFrom` (lines 650–671) | **Implemented** | -| `Block` | Full | Remote actor blocked us → removes from `ap_followers` (lines 736–744) | **Implemented** | -| `Add` / `Remove` | Full | No-op — logged only. Mastodon uses these for featured collection management (lines 745–750) | **Partial** — not used for featured collection sync | -| `Flag` | Full | Not handled | **Not implemented** — no report/moderation inbox | -| `EmojiReact` | Full (LitePub) | Not handled | **Not implemented** | -| `Dislike` | Full | Not handled | **Not implemented** — rarely used in fediverse | -| `Question` | Full | Not handled specially | **Not implemented** — polls not parsed | -| `Arrive` / `Travel` / `Join` / `Leave` | Full | Not handled | **Not implemented** — niche activity types | -| `Invite` / `Offer` | Full | Not handled | **Not implemented** | -| `Read` / `View` / `Listen` | Full | Not handled | **Not implemented** — niche | - -**Score: 13/21 activity types handled (62%), covering ~99% of real-world fediverse traffic** - ---- - -## 2. Outbound Activities - -Outbound activities are sent via `ctx.sendActivity()` from syndicator (`index.js`), interaction controllers (`lib/controllers/interactions-*.js`), compose (`lib/controllers/compose.js`), and messages (`lib/controllers/messages.js`). - -| Activity | Fedify Support | Our Implementation | Status | -|---|---|---|---| -| `Create(Note)` | Full | Via syndicator (`jf2ToAS2Activity()`) + DM compose (`submitMessageController`) | **Implemented** | -| `Create(Article)` | Full | Via syndicator (`jf2ToAS2Activity()`) | **Implemented** | -| `Like` | Full | Reader interaction button (`interactions-like.js:14–115`) | **Implemented** | -| `Undo(Like)` | Full | Unlike button (`interactions-like.js:121–229`) | **Implemented** | -| `Announce` | Full | Boost button (`interactions-boost.js:14–~100`) | **Implemented** | -| `Undo(Announce)` | Full | Unboost button (`interactions-boost.js:~101–~180`) | **Implemented** | -| `Follow` | Full | Reader follow + migration + batch refollow (`index.js:572–667`) | **Implemented** | -| `Undo(Follow)` | Full | Unfollow button (`index.js:674–~750`) | **Implemented** | -| `Accept(Follow)` | Full | Auto-accept on inbound Follow (`inbox-listeners.js:120–128`) | **Implemented** | -| `Update(Person)` | Full | Profile edit broadcasts to all followers (`index.js:761–~850`) | **Implemented** | -| `Delete` | Full | Not sent when posts are deleted | **Not implemented** | -| `Block` | Full | Local-only mute/block, no `Block` activity sent to remote | **Not implemented** | -| `Flag` | Full | No report sending UI | **Not implemented** | -| `Move` | Full | No outbound account migration | **Not implemented** | -| `Reject(Follow)` | Full | Auto-accept only, no manual approval/reject | **Not implemented** | -| `Create(Question)` | Full | No poll creation | **Not implemented** | - -**Score: 10/16 common outbound activities (63%)** - ---- - -## 3. Federation Dispatchers & Collections - -All dispatchers are registered in `lib/federation-setup.js`. - -| Dispatcher/Collection | Fedify Support | Our Implementation | Status | -|---|---|---|---| -| Actor (`Person`) | Full | Full with 5 actor types (Person, Service, Organization, Group, Application). Instance actor for shared inbox signing. RSA + Ed25519 key pairs. `mapHandle()` + `mapAlias()`. (lines 134–160) | **Implemented** | -| Inbox (personal + shared) | Full | Both endpoints registered. `setSharedKeyDispatcher()` for Authorized Fetch servers. (lines 283–295) | **Implemented** | -| Outbox | Full | Paginated, converts published blog posts to `Create(Note\|Article)` via `jf2ToAS2Activity()`. 20 per page. (lines 589–~650) | **Implemented** | -| Followers | Full | Paginated + one-shot mode for `sendActivity("followers")` batch delivery. Counter. (lines 396–445) | **Implemented** | -| Following | Full | Paginated with counter. 20 per page. (lines 447–475) | **Implemented** | -| Liked | Full | From `posts` collection where `post-type: "like"`. Paginated. (lines 477–518) | **Implemented** | -| Featured (pinned posts) | Full | Admin UI + AP collection. Converts pinned posts via `jf2ToAS2Activity()`. (lines 520–555) | **Implemented** | -| Featured Tags | Full | Admin UI + AP collection. Hashtag objects with category page links. (lines 557–587) | **Implemented** | -| Object dispatcher | Full | Content negotiation on individual post URLs. Returns `Create(Note\|Article)` AS2 JSON-LD for `Accept: application/activity+json`. | **Implemented** | -| WebFinger | Full | With OStatus subscribe link for remote follow from WordPress AP, Misskey, etc. (lines 275–282) | **Implemented** | -| NodeInfo | Full | Version 2.1. Reports software, protocols, total posts, active users. (lines 322–339) | **Implemented** | -| `.authorize()` on actor | Full | Intentionally disabled — causes infinite loops with Authorized Fetch servers. See CLAUDE.md Gotcha #16. | **Not implemented** | -| Custom collections | Full | Not used | **Not implemented** | - -**Score: 11/13 (85%)** - ---- - -## 4. Cryptography & Security - -Key storage in `ap_keys` collection. Key generation and signing handled by Fedify internals. - -| Feature | Fedify Support | Our Implementation | Status | -|---|---|---|---| -| RSA key pairs (HTTP Signatures) | Full | Generated on first use, stored as PEM in `ap_keys` (`federation-setup.js`) | **Implemented** | -| Ed25519 key pairs (Object Integrity Proofs) | Full | Generated on first use, stored as JWK in `ap_keys` | **Implemented** | -| HTTP Signatures (draft-cavage-12) | Full | Automatic via Fedify signing on all outbound requests | **Implemented** | -| HTTP Message Signatures (RFC 9421) | Full | Automatic via Fedify double-knocking negotiation | **Implemented** | -| Double-Knocking negotiation | Full | Automatic — Fedify caches per-server signature spec preference | **Implemented** | -| Authenticated Document Loader | Full | Used in all inbox handlers via `getAuthLoader()` helper. Required for Authorized Fetch servers (hachyderm.io, etc.) | **Implemented** | -| Object Integrity Proofs (FEP-8b32) | Full | Ed25519 keys stored; Fedify creates proofs automatically | **Implemented** (via Fedify) | -| Linked Data Signatures | Full | Fedify handles verification on inbound; not explicitly configured | **Partial** — verification only | -| Authorized Fetch on actor endpoint | Full | Disabled — `.authorize()` causes infinite key-fetch loops. Instance actor used as workaround for shared inbox signing. | **Not implemented** | -| Origin-based security (FEP-fe34) | Full | Not configured — using Fedify defaults | **Not implemented** | -| Inbox idempotency | Full | Not explicitly configured — using Fedify default (`"per-inbox"`) | **Implemented** (default) | -| Signature time window | Full | Default (1 hour) | **Implemented** (default) | -| CSRF protection | N/A (app concern) | Token generation + validation on all POST routes (`lib/csrf.js`) | **Implemented** | -| Content sanitization | N/A (app concern) | `sanitize-html` on all inbound content (`timeline-store.js`) | **Implemented** | - -**Score: 8/12 Fedify-specific features (67%)** - ---- - -## 5. Content & Object Types - -Object creation in `lib/jf2-to-as2.js`. Object parsing in `lib/timeline-store.js` and `lib/inbox-listeners.js`. - -| Object Type | Fedify Support | Our Implementation | Status | -|---|---|---|---| -| `Note` | Full | Create, display, syndicate. Primary post type for notes/replies/DMs. | **Implemented** | -| `Article` | Full | Create, display, syndicate. Used for article post type. | **Implemented** | -| `Image` (attachment) | Full | Photo posts with `Image` attachments in `jf2ToAS2Activity()` | **Implemented** | -| `Video` (attachment) | Full | Video post attachments | **Implemented** | -| `Audio` (attachment) | Full | Audio post attachments | **Implemented** | -| `Hashtag` (tag) | Full | Tags on syndicated posts, featured tags collection, tag timeline | **Implemented** | -| `Mention` (tag) | Full | Tags on replies for addressing, DM recipient mentions | **Implemented** | -| `PropertyValue` | Full | Profile attachment fields (name/value pairs) | **Implemented** | -| Quote posts (FEP-044f) | Full | Ingest via `quoteUrl` (3 namespaces), enrich via `fetchAndStoreQuote()`, render via `ap-quote-embed.njk` | **Implemented** | -| `Question` (polls) | Full | Not parsed — poll posts render without options | **Not implemented** | -| `Event` | Full | Not handled — events render as generic objects | **Not implemented** | -| `Page` | Full | Passthrough via content negotiation only | **Partial** | -| `ChatMessage` (LitePub DMs) | Full | Not handled — we use standard `Create(Note)` DM addressing | **Not implemented** | -| `Tombstone` | Full | Not created when posts are deleted | **Not implemented** | -| `Emoji` (custom) | Full (`toot:Emoji`) | Not handled — custom emoji renders as `:shortcode:` text | **Not implemented** | -| `Place` (location) | Full | Not handled — location data ignored | **Not implemented** | -| Sensitive / Content Warning | Full | `sensitive` flag displayed on inbound items but not settable on outbound | **Partial** | -| `Source` (original markup) | Full | Not used on outbound activities | **Not implemented** | - -**Score: 10/18 (56%), but core types fully covered** - ---- - -## 6. Audience Addressing & Visibility - -Addressing logic in `lib/jf2-to-as2.js` (lines 179–194) and `lib/controllers/messages.js` for DMs. - -| Visibility Mode | Fedify Support | Our Implementation | Status | -|---|---|---|---| -| **Public** (`to: PUBLIC_COLLECTION`, `cc: followers`) | Full | Standard addressing for all syndicated posts | **Implemented** | -| **Unlisted** (`to: followers`, `cc: PUBLIC_COLLECTION`) | Full | Not available — no UI option | **Not implemented** | -| **Followers-only** (`to: followers`, no PUBLIC) | Full | Not available — all posts are public | **Not implemented** | -| **Direct/DM** (`to: specific actors` only) | Full | Inbound detection (`isDirectMessage()`) + outbound via `submitMessageController` | **Implemented** | - -**Score: 2/4 (50%) — the two missing modes are rarely needed for IndieWeb sites** - ---- - -## 7. FEP (Fediverse Enhancement Proposals) - -| FEP | Description | Our Implementation | Status | -|---|---|---|---| -| FEP-8b32 | Object Integrity Proofs | Ed25519 keys generated and stored; Fedify creates proofs on outbound activities | **Implemented** (via Fedify) | -| FEP-521a | Multiple Cryptographic Keys | Both RSA + Ed25519 key pairs via `setKeyPairsDispatcher()` | **Implemented** | -| FEP-044f | Quote Posts | Full pipeline: ingest `quoteUrl` (3 namespaces), enrich via `fetchAndStoreQuote()`, render embedded card, strip inline `RE:` references | **Implemented** | -| FEP-8fcf | Followers Collection Synchronization | Not configured — `syncCollection` option not passed to `sendActivity()` | **Not implemented** | -| FEP-fe34 | Origin-Based Security | Not configured — using Fedify defaults (`crossOrigin` not set) | **Not implemented** | -| FEP-ae0c | Relay Protocols | `@fedify/relay` not used — personal site doesn't need relay | **Not implemented** | -| FEP-c0e0 | Actor Succession | `successor` property not set on actor | **Not implemented** | -| FEP-9091 | DID-Based Actor Identification | `DidService`/`Export` not used | **Not implemented** | -| FEP-5711 | Inverse Collection Properties | `likesOf`, `sharesOf`, etc. not exposed | **Not implemented** | - -**Score: 3/9 (33%) — the three implemented FEPs are the most impactful for interoperability** - ---- - -## 8. Infrastructure & Operations - -| Feature | Fedify Support | Our Implementation | Status | -|---|---|---|---| -| Redis message queue | `@fedify/redis` | `RedisMessageQueue` with `parallelWorkers` config (default 5) | **Implemented** | -| In-process queue | `InProcessMessageQueue` | Fallback when `redisUrl` not set | **Implemented** | -| MongoDB KvStore | Custom (app-provided) | `MongoKvStore` in `lib/kv-store.js` with `get`/`set`/`delete`/`list()` (required in Fedify 2.0) | **Implemented** | -| Debug dashboard | `@fedify/debugger` | Optional via `debugDashboard: true`, password-protected at `/{mount}/__debug__/` | **Implemented** | -| OpenTelemetry tracing | Full | Via `@fedify/debugger` `FedifySpanExporter` | **Implemented** | -| LogTape logging | Full | Configured once with `_logtapeConfigured` flag to prevent duplicate setup | **Implemented** | -| Delivery failure handling | Full | 404/410 permanent failures logged + stored in `ap_activities` (lines 344–361) | **Implemented** | -| Exponential backoff retry | Full | Using Fedify default retry policy | **Implemented** | -| Activity transformers | Full | Not used — `autoIdAssigner()` and `actorDehydrator()` defaults only | **Not implemented** | -| PostgreSQL queue | `@fedify/postgres` | Not applicable — using Redis | N/A | -| SQLite queue | `@fedify/sqlite` | Not applicable — using Redis | N/A | - -**Score: 8/9 relevant features (89%)** - ---- - -## 9. Application-Level Features (Beyond Fedify) - -These features are built on top of Fedify — Fedify provides the federation primitives, we provide the application logic. - -| Feature | Description | Status | -|---|---|---| -| Timeline reader | Full reader UI with tabs (notes, articles, boosts, media, replies, unread) | **Implemented** | -| Notifications | Like, boost, follow, mention, reply, DM notification types with unread tracking | **Implemented** | -| Direct messages | Inbound + outbound DMs with conversation sidebar, compose form | **Implemented** | -| Explore | Public Mastodon timeline aggregation from configured instances | **Implemented** | -| Hashtag explore | Cross-instance hashtag search via Mastodon API | **Implemented** | -| Tag timeline | Posts from followed accounts filtered by hashtag | **Implemented** | -| Post detail | Full post view with replies, quote enrichment | **Implemented** | -| Remote profile | View remote actor profiles with follow/mute/block actions | **Implemented** | -| Moderation | Mute (by URL or keyword), block, with filtering across all views | **Implemented** | -| Mastodon migration | CSV import + WebFinger resolution + batch re-follow state machine | **Implemented** | -| Featured posts | Pin/unpin posts to featured collection | **Implemented** | -| Featured tags | Manage featured hashtags | **Implemented** | -| Profile editor | Name, summary, icon, image, attachments, broadcasts update to followers | **Implemented** | -| Link previews | Open Graph unfurling via `unfurl.js` for timeline items | **Implemented** | -| Infinite scroll | Unified Alpine.js component with configurable cursor parameters | **Implemented** | -| CSRF protection | Token generation/validation on all POST routes | **Implemented** | -| Content sanitization | `sanitize-html` on all inbound content | **Implemented** | -| Activity log | Full inbound/outbound activity logging with TTL cleanup | **Implemented** | -| Timeline cleanup | Retention-based pruning (`timelineRetention` config) | **Implemented** | -| Hashtag following | Follow/unfollow hashtags, items from non-followed accounts matching tags appear in timeline | **Implemented** | -| Public profile page | HTML fallback for actor URL when accessed from browser | **Implemented** | - ---- - -## 10. Overall Summary - -| Category | Score | Percentage | Notes | -|---|---|---|---| -| Inbound Activities | 13/21 | 62% | All high-traffic types covered | -| Outbound Activities | 10/16 | 63% | Missing: Delete, Block, Flag, Move, Reject | -| Dispatchers/Collections | 11/13 | 85% | Near complete | -| Crypto/Security | 8/12 | 67% | Core signing works | -| Object Types | 10/18 | 56% | Core types done | -| Addressing | 2/4 | 50% | Public + DM only | -| FEPs | 3/9 | 33% | Key FEPs implemented | -| Infrastructure | 8/9 | 89% | Excellent | -| **Weighted Overall** | — | **~70%** | **~95%+ of real-world fediverse traffic covered** | - ---- - -## 11. Gap Analysis: High-Impact Improvements - -Ordered by impact-to-effort ratio. - -### Priority 1 — High Impact, Low Effort - -| Gap | Impact | Effort | Details | -|---|---|---|---| -| **Outbound `Delete` activity** | High | Low | When a post is deleted in Indiekit, remote servers are never notified. The post remains visible on all federated instances indefinitely. Hook into Indiekit's post delete lifecycle, send `Delete(Tombstone)` to followers. | -| **Outbound `Block` activity** | Medium | Low | Our block is local-only (`ap_blocked`). Remote servers don't know we blocked them, so they continue delivering activities. Send `Block` activity on block, `Undo(Block)` on unblock. | -| **Unlisted addressing mode** | Medium | Low | Add a "visibility" option to the syndicator: public (default), unlisted (`to: followers, cc: PUBLIC`). Useful for posts that shouldn't appear on public timelines but are still accessible via link. | - -### Priority 2 — Medium Impact, Medium Effort - -| Gap | Impact | Effort | Details | -|---|---|---|---| -| **Question/Poll support (inbound)** | Medium | Medium | Poll posts from Mastodon render without options. Parse `Question` object's `inclusiveOptions`/`exclusiveOptions`, display vote options and results in timeline. Voting (outbound) is a separate feature. | -| **`Flag` handler (inbound reports)** | Medium | Medium | Other servers can't send us abuse reports. Add `Flag` inbox listener, store in a `ap_reports` collection, add moderation UI tab. | -| **Content Warning / Sensitive flag (outbound)** | Medium | Low | Inbound sensitive content is displayed with a warning. Add a "sensitive" / CW option to the compose form and syndicator so outbound posts can include content warnings. | -| **Followers-only addressing** | Medium | Medium | Add a "followers-only" visibility option. Requires `to: followers` only, no PUBLIC. Also needs consideration for who can see the post on our own site. | - -### Priority 3 — Low Impact - -| Gap | Impact | Effort | Details | -|---|---|---|---| -| **Custom Emoji** | Low | Medium | Mastodon custom emoji renders as `:shortcode:` text. Parse `Emoji` tags, fetch images, inline-replace in content. | -| **`Reject(Follow)` / manual approval** | Low | Medium | Currently all follows are auto-accepted. Add a "manually approves followers" mode with pending/accept/reject UI. | -| **`Tombstone` on delete** | Low | Low | Instead of just deleting from collections, create a `Tombstone` object for the deleted resource. Mostly a federation correctness improvement. | -| **Activity transformers** | Low | Low | Fedify's `actorDehydrator()` improves Threads compatibility. Consider enabling for broader compatibility. | -| **FEP-8fcf Followers Sync** | Low | Low | Pass `syncCollection: true` to `sendActivity()` calls. Reduces duplicate deliveries for servers that support it. | -| **FEP-fe34 Origin-Based Security** | Low | Low | Set `crossOrigin: "ignore"` or `"throw"` on federation options. Prevents spoofed attribution attacks. | - -### Not Recommended (Skip) - -| Gap | Reason | -|---|---| -| `EmojiReact` | Misskey/Pleroma-only, very niche | -| `Arrive`/`Travel`/`Join`/`Leave` | Almost never seen in real fediverse | -| `Invite`/`Offer` | Group-specific, very niche | -| `Dislike` | Not implemented by any major fediverse software | -| Relay support (FEP-ae0c) | Only useful at scale, not for personal sites | -| DID-based identity (FEP-9091) | Future spec, minimal adoption | -| Actor succession (FEP-c0e0) | Future spec, minimal adoption | -| `ChatMessage` (LitePub DMs) | Our standard DM addressing works with all servers | - ---- - -## 12. Data Flow Reference - -### Outbound Activity Flow - -``` -Indiekit blog post (JF2) - ↓ -syndicator.syndicate() [index.js] - ↓ -jf2ToAS2Activity() [lib/jf2-to-as2.js — converts JF2 → Fedify vocab objects] - ↓ -ctx.sendActivity({ identifier: handle }, "followers", activity) [Fedify] - ↓ -Redis queue [or InProcessMessageQueue] - ↓ -HTTP POST to follower inboxes [signed with RSA/Ed25519 by Fedify] -``` - -### Inbound Activity Flow - -``` -Remote server HTTP POST to /{mount}/inbox [HTTP Signature verified by Fedify] - ↓ -federation-bridge.js [reconstructs body if Express consumed stream, uses req.originalUrl] - ↓ -Fedify matches activity type → calls registered listener - ↓ -inbox-listeners.js [authenticated document loader for all remote fetches] - ↓ -MongoDB storage [ap_followers, ap_timeline, ap_notifications, ap_messages, ap_activities] - ↓ -Admin UI renders data [reader, notifications, messages, moderation] -``` - -### Reader Timeline Pipeline - -``` -Raw items from ap_timeline - ↓ -applyTabFilter() [notes/articles/boosts/media/replies — lib/item-processing.js] - ↓ -loadModerationData() [load muted URLs, keywords, blocked URLs] - ↓ -postProcessItems() [filter muted/blocked, strip quote refs, build interaction map] - ↓ -renderItemCards() [server-side Nunjucks → HTML for AJAX responses] - ↓ -Alpine.js infinite scroll [apInfiniteScroll component — assets/reader-infinite-scroll.js] -``` - ---- - -## 13. MongoDB Collections Reference - -| Collection | Records | Indexes | TTL | -|---|---|---|---| -| `ap_followers` | Accounts following us | `actorUrl` (unique) | No | -| `ap_following` | Accounts we follow | `actorUrl` (unique) | No | -| `ap_activities` | Activity log | `direction`, `type`, `actorUrl`, `objectUrl`, `receivedAt` | Yes (`activityRetentionDays`, default 90) | -| `ap_keys` | Crypto key pairs | `type` (rsa/ed25519) | No | -| `ap_kv` | Fedify KV store | `_id` (key path) | Yes (Fedify-managed) | -| `ap_profile` | Actor profile (single doc) | — | No | -| `ap_featured` | Pinned posts | `postUrl` | No | -| `ap_featured_tags` | Featured hashtags | `tag` | No | -| `ap_timeline` | Reader timeline | `uid` (unique), `published`, `author.url`, `type` | No (manual cleanup via `timelineRetention`) | -| `ap_notifications` | Notifications | `uid` (unique), `type`, `read`, `createdAt` | Yes (`notificationRetentionDays`, default 30) | -| `ap_messages` | Direct messages | `uid` (unique), `conversationId`+`published`, `read`, `direction` | Yes (reuses `notificationRetentionDays`) | -| `ap_muted` | Muted actors/keywords | `url` or `keyword` | No | -| `ap_blocked` | Blocked actors | `url` | No | -| `ap_interactions` | Like/boost tracking | `objectUrl`, `type` | No | -| `ap_followed_tags` | Hashtags we follow | `tag` | No | - ---- - -## 14. Configuration Reference - -```javascript -{ - mountPath: "/activitypub", // URL prefix for all routes - actor: { - handle: "rick", // Fediverse username (@rick@rmendes.net) - name: "Ricardo Mendes", // Display name (seeds profile on first run) - summary: "", // Bio (seeds profile) - icon: "", // Avatar URL (seeds profile) - }, - checked: true, // Syndicator checked by default in Micropub UI - alsoKnownAs: "", // Mastodon migration alias (for Move activities) - activityRetentionDays: 90, // TTL for ap_activities (0 = forever) - storeRawActivities: false, // Store full JSON of inbound activities - redisUrl: "", // Redis for delivery queue (empty = in-process) - parallelWorkers: 5, // Parallel delivery workers (with Redis) - actorType: "Person", // Person | Service | Organization | Group | Application - logLevel: "warning", // Fedify log level: debug | info | warning | error | fatal - timelineRetention: 1000, // Max timeline items (0 = unlimited) - notificationRetentionDays: 30, // Days to keep notifications (0 = forever) - debugDashboard: false, // Enable @fedify/debugger at {mount}/__debug__/ - debugPassword: "", // Password for debug dashboard -} -``` - ---- - -*This audit reflects the state of the plugin at version 2.9.2. It should be updated when new features are added or when Fedify releases new capabilities.* diff --git a/docs/plans/2026-02-28-explore-tabbed-redesign.md b/docs/plans/2026-02-28-explore-tabbed-redesign.md deleted file mode 100644 index f2bdb92..0000000 --- a/docs/plans/2026-02-28-explore-tabbed-redesign.md +++ /dev/null @@ -1,513 +0,0 @@ -# Explore Page Tabbed Redesign Implementation Plan - -Created: 2026-02-28 -Status: VERIFIED -Approved: Yes -Iterations: 0 -Worktree: No - -> **Status Lifecycle:** PENDING → COMPLETE → VERIFIED -> **Iterations:** Tracks implement→verify cycles (incremented by verify phase) -> -> - PENDING: Initial state, awaiting implementation -> - COMPLETE: All tasks implemented -> - VERIFIED: All checks passed -> -> **Approval Gate:** Implementation CANNOT proceed until `Approved: Yes` -> **Worktree:** No — working directly on current branch - -## Summary - -**Goal:** Replace the cramped deck/column layout on the ActivityPub explore page with a full-width tabbed design. Three tab types: Search (always first, not removable), Instance (pinned instances with local/federated badge), and Hashtag (aggregated across all pinned instances). New `ap_explore_tabs` collection replaces `ap_decks` (clean start, no migration). - -**Architecture:** Server-rendered tab navigation with Alpine.js for tab content loading. Each tab loads its own timeline via the existing explore API (instance tabs) or a new hashtag aggregation API (hashtag tabs). The tab bar is a horizontal scrollable row with the Search tab always first, followed by user-ordered Instance and Hashtag tabs. Tab reordering uses server-side PATCH endpoint. No limit on tab count. - -**Tech Stack:** Express routes (Node.js), Nunjucks templates, Alpine.js 3.x for client-side interactivity, MongoDB for `ap_explore_tabs` collection, Mastodon API v1 for timelines. - -## Scope - -### In Scope - -- Replace deck grid layout with full-width tab navigation -- New `ap_explore_tabs` collection with schema: `{ type, domain?, scope?, hashtag?, order, addedAt }` -- Search tab: existing instance search + optional hashtag field switching between `/timelines/public` and `/timelines/tag/{hashtag}` -- Instance tabs: full-width timeline with local/federated scope badge -- Hashtag tabs: parallel queries across all pinned instances, merge by date, dedup by post URL -- Tab CRUD API: add, remove, reorder -- Tab reordering UI: up/down arrow buttons (simpler, more accessible than drag-and-drop) -- Each tab loads independently with infinite scroll -- Replace deck-related Alpine.js components with tab-based ones -- Replace deck CSS with tab CSS -- Update i18n locale strings -- Note: The responsive CSS fix (`width: 100%` + `box-sizing: border-box` on `.ap-lookup__input` and `.ap-explore-form__input`) was already committed prior to this plan — no action needed - -### Out of Scope - -- Drag-and-drop tab reordering (deferred — up/down arrows first, DnD can be added later) -- Per-instance hashtag filter within instance tabs (deferred per user decision) -- Migration of old `ap_decks` data (clean start per user decision) -- Changes to the main reader timeline, tag timeline, or notifications - -## Prerequisites - -- Node.js >= 22 (already in place) -- `@rmdes/indiekit-endpoint-activitypub` repo at version 2.0.36 -- MongoDB with existing ActivityPub collections - -## Context for Implementer - -> This section is critical for cross-session continuity. - -- **Patterns to follow:** - - Controller pattern: `lib/controllers/explore.js` — exports factory functions `controllerName(mountPath)` returning `async (request, response, next) => { ... }` - - API JSON endpoint pattern: `exploreApiController()` at `explore.js:260` — renders partials server-side via `request.app.render()`, returns `{ html, maxId }` - - CSRF validation pattern: `lib/controllers/decks.js:45` — `validateToken(request)` from `../csrf.js` - - SSRF prevention: `validateInstance()` at `explore.js:22` — validates hostnames, blocks private IPs - - Alpine.js registration: `reader-decks.js:9` — `document.addEventListener("alpine:init", () => { Alpine.data(...) })` - - CSS conventions: Uses Indiekit theme custom properties (`--color-on-background`, `--color-primary`, etc.) - - Tab styling: Existing `.ap-tabs` / `.ap-tab` / `.ap-tab--active` CSS at `reader.css:91-134` (reused and extended) - -- **Conventions:** - - ESM modules (`import`/`export`) - - Dates stored as ISO 8601 strings: `new Date().toISOString()` - - Template variables must avoid collisions with Nunjucks macro names imported in `default.njk` (e.g., `tag` collides with the `tag` macro — use `hashtag` instead) - - Express 5: No `redirect("back")` — use explicit paths - - sanitize-html for any remote content displayed in HTML - -- **Key files:** - - `index.js` — Plugin entry; collection registration (line 888), route registration (line 239-246), index creation (line 1036-1039) - - `lib/controllers/explore.js` — Current explore controller (405 lines) with `exploreController`, `exploreApiController`, `instanceSearchApiController`, `instanceCheckApiController`, `popularAccountsApiController`, and helper `mapMastodonStatusToItem` - - `lib/controllers/decks.js` — Current deck CRUD (137 lines): `listDecksController`, `addDeckController`, `removeDeckController` - - `views/activitypub-explore.njk` — Current explore template (218 lines) with Search tab and Decks tab - - `assets/reader-decks.js` — Alpine components: `apDeckToggle`, `apDeckColumn` (212 lines) - - `assets/reader-infinite-scroll.js` — Alpine components: `apExploreScroll`, `apInfiniteScroll` (183 lines) - - `assets/reader-autocomplete.js` — Alpine components: `apInstanceSearch`, `apPopularAccounts` (214 lines) - - `assets/reader.css` — All styles (2248 lines); deck styles at lines 2063-2248 - - `locales/en.json` — i18n strings; explore section at line 229 - -- **Gotchas:** - - Template variable `tag` is shadowed by Nunjucks macro from `default.njk` — always use `hashtag` in template context - - The `.ap-tabs` CSS class already exists and is used for the current Search/Decks tab bar — it will be extended for the new design - - `reader-infinite-scroll.js` contains `apExploreScroll` (for explore page) AND `apInfiniteScroll` (for main reader timeline) — only the former is being replaced - - The `ap_kv` collection is used for FediDB caching — not related to deck/tab storage - - Mastodon hashtag timeline API: `GET /api/v1/timelines/tag/{hashtag}?local=true|false&limit=20&max_id=X` — public, no auth needed - -- **Domain context:** - - The explore page lets users browse public timelines from remote Mastodon-compatible instances - - Instance tabs pin specific instances so users don't re-search each time - - Hashtag tabs aggregate a hashtag across ALL pinned instances in parallel (e.g., #indieweb from mastodon.social + fosstodon.org + ...) - - The Search tab is the entry point for discovering new instances + one-off browsing - -## Runtime Environment - -- **Start command:** Deployed via Cloudron (`/app/pkg/start.sh`); locally via `node --loader` or through Indiekit dev server -- **Port:** 8080 (Indiekit), 3000 (nginx proxy) -- **Deploy path:** `indiekit-cloudron/Dockerfile` installs from npm, `cloudron build --no-cache && cloudron update --app rmendes.net --no-backup` -- **Health check:** `curl -s https://rmendes.net/activitypub/ | head -20` (should return dashboard HTML) -- **Restart procedure:** `cloudron restart --app rmendes.net` or full rebuild cycle - -## Feature Inventory — Files Being Replaced - -This is a **refactoring task** — the deck system is being replaced by a tab system. - -### Files Being Replaced - -| Old File | Functions/Features | Mapped to Task | -| --- | --- | --- | -| `lib/controllers/decks.js` | `listDecksController()`, `addDeckController()`, `removeDeckController()` — CRUD for `ap_decks` | Task 2 (replaced by tab CRUD) | -| `lib/controllers/explore.js` | `exploreController()` — renders explore page with deck data | Task 3, Task 5 | -| `lib/controllers/explore.js` | `exploreApiController()` — AJAX infinite scroll for single instance | Task 5, Task 6 | -| `lib/controllers/explore.js` | `instanceSearchApiController()` — FediDB autocomplete | Task 3 (kept as-is) | -| `lib/controllers/explore.js` | `instanceCheckApiController()` — timeline support check | Task 3 (kept as-is) | -| `lib/controllers/explore.js` | `popularAccountsApiController()` — popular accounts API | Task 3 (kept as-is) | -| `lib/controllers/explore.js` | `validateInstance()` — SSRF-safe hostname validation | Task 2 (reused as-is) | -| `lib/controllers/explore.js` | `mapMastodonStatusToItem()` — status-to-timeline-item mapping | Task 5, Task 6 (reused as-is) | -| `views/activitypub-explore.njk` | Search form + autocomplete | Task 3 | -| `views/activitypub-explore.njk` | Deck grid + deck columns | Task 4 | -| `views/activitypub-explore.njk` | Instance timeline + infinite scroll | Task 5 | -| `assets/reader-decks.js` | `apDeckToggle` — star/add-to-deck button | Task 4 (replaced by "Pin" button) | -| `assets/reader-decks.js` | `apDeckColumn` — individual deck column with infinite scroll | Task 5 (replaced by tab panel) | -| `assets/reader-infinite-scroll.js` | `apExploreScroll` — explore page infinite scroll | Task 5 (replaced by tab-scoped scroll) | -| `assets/reader-infinite-scroll.js` | `apInfiniteScroll` — main reader timeline scroll | NOT TOUCHED (kept as-is) | -| `assets/reader-autocomplete.js` | `apInstanceSearch` — instance autocomplete | Task 3 (extended with hashtag field) | -| `assets/reader-autocomplete.js` | `apPopularAccounts` — popular account autocomplete | NOT TOUCHED (kept as-is) | -| `assets/reader.css` | `.ap-deck-*` styles (lines 2063-2248) | Task 4 (replaced by tab styles) | -| `assets/reader.css` | `.ap-explore-deck-toggle` styles (lines 2063-2103) | Task 4 (replaced by "Pin" styles) | -| `assets/reader.css` | `.ap-tabs` styles (lines 91-134) | Task 4 (extended for dynamic tabs) | -| `locales/en.json` | `explore.tabs.*`, `explore.deck.*` strings | Task 3 (updated strings) | -| `index.js` | `ap_decks` collection registration + indexes (line 888, 1036-1039) | Task 1 | -| `index.js` | Deck route registration (lines 244-246) | Task 2 | - -### Feature Mapping Verification - -- [x] All old files listed above -- [x] All functions/classes identified -- [x] Every feature has a task number -- [x] No features accidentally omitted - -## Progress Tracking - -**MANDATORY: Update this checklist as tasks complete. Change `[ ]` to `[x]`.** - -- [x] Task 1: Collection setup — `ap_explore_tabs` replaces `ap_decks` -- [x] Task 2: Tab CRUD API — add, remove, reorder endpoints -- [x] Task 3: Search tab — form with hashtag field, updated template -- [x] Task 4: Tab bar UI — dynamic tabs with scope badges, reordering, pin button -- [x] Task 5: Instance tab panel — full-width timeline with infinite scroll -- [x] Task 6: Hashtag tab panel — cross-instance aggregation -- [x] Task 7: Cleanup — remove old deck code, update CSS, update locales - -**Total Tasks:** 7 | **Completed:** 7 | **Remaining:** 0 - -## Implementation Tasks - -### Task 1: Collection Setup — `ap_explore_tabs` Replaces `ap_decks` - -**Objective:** Register the new `ap_explore_tabs` MongoDB collection with proper indexes, replacing `ap_decks`. Clean start — no migration of old data. - -**Dependencies:** None - -**Files:** - -- Modify: `index.js` — Replace `ap_decks` collection registration with `ap_explore_tabs`, update `this._collections`, create indexes - -**Key Decisions / Notes:** - -- Schema: `{ type: "instance"|"hashtag", domain?: string, scope?: "local"|"federated", hashtag?: string, order: number, addedAt: string (ISO 8601) }` -- Indexes: single unique compound index on `(type, domain, scope, hashtag)`. **CRITICAL: All insertions MUST explicitly set ALL four fields** — instance tabs set `hashtag: null`, hashtag tabs set `domain: null, scope: null`. MongoDB treats missing fields and explicit `null` differently in compound indexes, so omitting a field would bypass the uniqueness constraint and allow duplicates. -- `order` field: integer, used for user-controlled tab ordering. New tabs get `order = max(existing orders) + 1`. Use `findOneAndUpdate` with `sort: { order: -1 }` to atomically determine the next order value (prevents race conditions on concurrent additions). -- No limit on tab count (removed the `MAX_DECKS = 8` restriction) -- Remove old `ap_decks` references from collection registration and `this._collections` - -**Definition of Done:** - -- [ ] `ap_explore_tabs` collection registered in `index.js` (replacing `ap_decks`) -- [ ] `this._collections` includes `ap_explore_tabs` instead of `ap_decks` -- [ ] Unique compound index on `(type, domain, scope, hashtag)` created -- [ ] Index on `order` field created for efficient sorting -- [ ] Old `ap_decks` references completely removed from `index.js` - -**Verify:** - -- `grep -r "ap_decks" index.js` returns nothing -- `grep "ap_explore_tabs" index.js` shows collection registration, `this._collections`, and index creation - ---- - -### Task 2: Tab CRUD API — Add, Remove, Reorder Endpoints - -**Objective:** Create the tab management API replacing the old deck CRUD. Supports adding instance tabs, adding hashtag tabs, removing any tab, and reordering tabs. - -**Dependencies:** Task 1 - -**Files:** - -- Create: `lib/controllers/tabs.js` — New tab CRUD controller with `listTabsController`, `addTabController`, `removeTabController`, `reorderTabsController`, and `validateHashtag()` helper -- Modify: `index.js` — Replace deck route imports and registrations with tab routes -- Delete: `lib/controllers/decks.js` — Deleted here when replaced by tabs.js (Task 7 only verifies it's gone) - -**Key Decisions / Notes:** - -- `POST /admin/reader/api/tabs` — Add tab. Body: `{ type: "instance"|"hashtag", domain?, scope?, hashtag? }`. Validates domain via `validateInstance()`, validates hashtag via `validateHashtag()`. Auto-assigns `order = max(existing) + 1`. **CRITICAL: Insertions MUST explicitly set all four indexed fields** — instance tabs: `{ type, domain, scope, hashtag: null, order, addedAt }`, hashtag tabs: `{ type, domain: null, scope: null, hashtag, order, addedAt }`. -- `POST /admin/reader/api/tabs/remove` — Remove tab. Body: `{ type, domain?, scope?, hashtag? }`. After removal, re-compacts order numbers to avoid gaps. -- `PATCH /admin/reader/api/tabs/reorder` — Reorder tabs. Body: `{ tabIds: [id1, id2, ...] }` — array of MongoDB `_id` strings in desired order. Sets `order = index` for each. -- `GET /admin/reader/api/tabs` — List all tabs sorted by `order` ascending. -- **`validateHashtag()` helper** (new, alongside `validateInstance()`): (1) Strip leading `#` characters, (2) Reject if empty after stripping, (3) Validate against `/^[\w]+$/` (alphanumeric + underscore only — matching Mastodon's hashtag rules), (4) Enforce max length of 100 chars. Call this in the add-tab endpoint for hashtag tabs AND in the hashtag explore endpoint (Task 6). -- All POST/PATCH endpoints require CSRF token validation via `validateToken(request)`. -- All domain inputs validated via `validateInstance()` (imported from explore.js). -- All tab routes registered in the `routes` getter (not `routesPublic`) to ensure IndieAuth authentication protects them. -- Reuse the existing CSRF and validation patterns from `decks.js`. - -**Definition of Done:** - -- [ ] `GET /admin/reader/api/tabs` returns all tabs sorted by order -- [ ] `POST /admin/reader/api/tabs` with `{ type: "instance", domain: "mastodon.social", scope: "local" }` creates a tab -- [ ] `POST /admin/reader/api/tabs` with `{ type: "hashtag", hashtag: "indieweb" }` creates a tab -- [ ] Duplicate tabs rejected with 409 -- [ ] `POST /admin/reader/api/tabs/remove` removes tab and re-compacts order -- [ ] `PATCH /admin/reader/api/tabs/reorder` updates order for all specified tabs -- [ ] `validateHashtag()` helper rejects empty, non-alphanumeric, and >100 char hashtags -- [ ] Hashtag tabs insert with explicit `domain: null, scope: null`; instance tabs insert with explicit `hashtag: null` -- [ ] CSRF validation on all mutating endpoints -- [ ] SSRF validation on domain inputs -- [ ] Old deck routes removed from `index.js` -- [ ] `lib/controllers/decks.js` deleted - -**Verify:** - -- `grep -r "ap_decks\|decks.js" index.js` returns nothing -- `grep "tabs.js\|api/tabs" index.js` shows new routes - ---- - -### Task 3: Search Tab — Form with Hashtag Field - -**Objective:** Update the Search tab's search form to add an optional hashtag field. When a hashtag is entered, the API call switches from `/timelines/public` to `/timelines/tag/{hashtag}`. Update both `exploreController()` (initial page load) and `exploreApiController()` (AJAX infinite scroll) to handle the `hashtag` query param. - -**Dependencies:** Task 1 - -**Files:** - -- Modify: `views/activitypub-explore.njk` — Add hashtag input field to search form (within the Search tab content section only; leave the tab nav bar unchanged — Task 4 replaces it entirely). Pass hashtag value to infinite scroll data attributes. -- Modify: `lib/controllers/explore.js` — Add `hashtag` query param handling in BOTH `exploreController()` AND `exploreApiController()`; change API URL construction to use `/timelines/tag/{hashtag}` when hashtag is provided; update template variables (remove deck references). Validate hashtag via `validateHashtag()` from `tabs.js`. -- Modify: `assets/reader-autocomplete.js` — Extend `apInstanceSearch` to handle hashtag field state -- Modify: `locales/en.json` — Update explore locale strings (remove deck strings, add tab/hashtag strings) - -**Key Decisions / Notes:** - -- The hashtag field is a plain text input next to the instance field. When filled, the explore API fetches `/api/v1/timelines/tag/{encodedHashtag}?local=true|false` instead of `/api/v1/timelines/public` -- The `hashtag` parameter is stripped of leading `#` and URL-encoded -- **Both `exploreController` and `exploreApiController` must handle the hashtag param** — without this, infinite scroll on search tab with hashtag would revert to the public timeline after the first page -- The template's infinite scroll data attributes must pass the hashtag value so the AJAX endpoint receives it on subsequent pages -- The Search tab is always the first tab and cannot be removed -- **Do NOT modify the tab navigation bar** (lines 14-24 of template) — leave it as-is in this task; Task 4 replaces it entirely with the dynamic tab bar. Only modify the Search tab content section. -- Keep `instanceSearchApiController`, `instanceCheckApiController`, `popularAccountsApiController` unchanged in explore.js -- Remove `decks`, `deckCount`, `isInDeck` from the controller's template variables - -**Definition of Done:** - -- [ ] Search form has an optional hashtag text input field -- [ ] When hashtag is provided, `exploreController` fetches from `/timelines/tag/{hashtag}` instead of `/timelines/public` -- [ ] When hashtag is provided, `exploreApiController` also fetches from `/timelines/tag/{hashtag}` (infinite scroll stays in hashtag mode) -- [ ] Hashtag is validated via `validateHashtag()`, URL-encoded, and stripped of leading `#` -- [ ] Scope radio buttons still work with hashtag mode -- [ ] Infinite scroll data attributes pass the hashtag value to the AJAX endpoint -- [ ] i18n strings updated: deck strings removed, hashtag placeholder string added - -**Verify:** - -- Open explore page — Search tab renders with instance + hashtag fields -- Submit with instance `mastodon.social` + hashtag `indieweb` — results load from tag timeline -- Scroll down — infinite scroll continues fetching from tag timeline (not reverting to public) -- Submit with instance only (no hashtag) — results load from public timeline (unchanged behavior) - ---- - -### Task 4: Tab Bar UI — Dynamic Tabs with Scope Badges, Reordering, Pin Button - -**Objective:** Build the dynamic tab bar that shows Search + user-added Instance/Hashtag tabs. Each Instance tab shows domain + scope badge. Each Hashtag tab shows `#tag`. Tabs have close buttons and up/down reorder arrows. Replace the star/deck-toggle button with a "Pin as tab" button on search results. Add UI for creating hashtag tabs. - -**Dependencies:** Task 2, Task 3 - -**Files:** - -- Modify: `views/activitypub-explore.njk` — Replace static Search/Decks tab nav with dynamic tab bar; add "Pin as tab" button for search results; add "Add hashtag tab" UI -- Create: `assets/reader-tabs.js` — Alpine.js component `apExploreTabs` for tab management (switching, adding, removing, reordering). **Guard init with DOM check:** `if (!document.querySelector('.ap-explore-tabs')) return;` — since the script loads on all reader pages via the shared layout, this prevents console errors on non-explore pages. -- Modify: `assets/reader.css` — Remove `.ap-deck-*` styles, add `.ap-explore-tab-*` styles for dynamic tabs with badges, controls, and overflow handling -- Modify: `views/layouts/ap-reader.njk` — Replace `reader-decks.js` script tag with `reader-tabs.js`. IMPORTANT: `reader-tabs.js` must load BEFORE the Alpine CDN script (same `defer` pattern as existing component scripts) since it registers Alpine data components via the `alpine:init` event. - -**Key Decisions / Notes:** - -- Tab bar: horizontal scrollable row. Search tab first (no close button, no reorder). All other tabs (Instance + Hashtag) sorted by `order` field **regardless of type** — tabs are freely interleaved, not grouped by type. -- Instance tab label: `{domain}` with a colored scope badge (local = blue, federated = purple) — reuse `.ap-deck-column__scope-badge` colors -- Hashtag tab label: `#{hashtag}` -- Each non-Search tab has: close button (×) and up/down arrows for reordering -- "Pin as tab" button replaces the star/deck-toggle button in search results area -- **"Add hashtag tab" UI:** A `+#` button at the end of the tab bar opens a small inline form (text input + confirm button) to add a hashtag tab. On submit, calls `POST /admin/reader/api/tabs` with `{ type: "hashtag", hashtag: value }`. The new tab appears in the tab bar with `#{hashtag}` label. -- When a tab is clicked, the tab content area switches to show that tab's timeline (Alpine.js handles visibility) -- The `apExploreTabs` Alpine component manages: active tab state, tab list from server, add/remove/reorder API calls -- Tab data is loaded via `GET /admin/reader/api/tabs` on page init -- **Reorder debouncing:** Debounce reorder API calls (500ms after last arrow click) so rapid clicks batch into a single request. This prevents race conditions from rapid successive clicks. -- **Tab bar overflow:** When tabs overflow horizontally, show fade gradients at edges to indicate scrollable content. Tab labels use `text-overflow: ellipsis` with `max-width: 150px` to truncate long domain names. Reorder arrows only visible on hover (desktop) or on long-press (mobile) to save space. -- **Accessibility (WAI-ARIA Tabs Pattern):** Tab bar uses `role="tablist"`, each tab uses `role="tab"` with `aria-selected`, `aria-controls` pointing to tab panel. Tab panels use `role="tabpanel"`. Arrow keys navigate between tabs. -- **CSRF 403 handling:** When a tab API call returns 403, show a clear error: "Session expired — please refresh the page." This handles stale CSRF tokens on long-lived pages. - -**Definition of Done:** - -- [ ] Tab bar shows Search tab + all user-created tabs from `ap_explore_tabs` -- [ ] Instance tabs display domain + colored scope badge (local=blue, federated=purple) -- [ ] Hashtag tabs display `#{hashtag}` -- [ ] Clicking a tab switches the visible content panel -- [ ] Close button (×) on non-Search tabs calls remove API and removes tab from bar -- [ ] Up/down arrows on non-Search tabs call reorder API and move tab in bar (debounced 500ms) -- [ ] "Pin as tab" button in search results adds instance+scope to tabs via add API -- [ ] "Add hashtag tab" button (`+#`) opens inline form to add a hashtag tab -- [ ] Tab bar overflow: fade gradients at edges, ellipsis on long labels -- [ ] Tab bar follows WAI-ARIA Tabs Pattern (role=tablist, role=tab, role=tabpanel, aria-selected, aria-controls) -- [ ] Tab bar is keyboard-navigable (arrow keys between tabs) -- [ ] Alpine component guarded with DOM check for non-explore pages -- [ ] Old `.ap-deck-*` CSS removed, new tab styles added -- [ ] `reader-decks.js` script tag replaced with `reader-tabs.js` in layout - -**Verify:** - -- Open explore page — tab bar visible with Search tab -- Browse an instance in Search tab — "Pin as tab" button appears -- Click "Pin as tab" — new Instance tab appears in tab bar with scope badge -- Click `+#` button — hashtag input form appears, enter "indieweb", confirm — hashtag tab appears as `#indieweb` -- Click the Instance tab — content area switches (empty initially, loaded in Task 5) -- Close button removes the tab -- Up/down arrows reorder tabs (verify via page reload) -- Open a non-explore reader page (e.g., timeline) — no console errors from reader-tabs.js - ---- - -### Task 5: Instance Tab Panel — Full-Width Timeline with Infinite Scroll - -**Objective:** When an Instance tab is active, load and display the full-width timeline from that instance with infinite scroll. Reuses `mapMastodonStatusToItem()` and the `exploreApiController` pattern. - -**Dependencies:** Task 4 - -**Files:** - -- Modify: `views/activitypub-explore.njk` — Add instance tab panel template section (conditionally visible based on active tab) -- Modify: `assets/reader-tabs.js` — Add timeline loading logic to `apExploreTabs` component for instance tab activation (fetch + render + infinite scroll) -- Modify: `lib/controllers/explore.js` — Ensure `exploreApiController` works for tab-driven requests (may need to accept hashtag param for search tab hashtag mode — already handled in Task 3) - -**Key Decisions / Notes:** - -- When an instance tab becomes active, if it hasn't loaded yet, fetch the first page from `GET /admin/reader/api/explore?instance={domain}&scope={scope}` -- Infinite scroll uses IntersectionObserver on a sentinel element within the tab panel (same pattern as `apDeckColumn` in `reader-decks.js:100-118`) -- **Tab content cache:** Cached in Alpine state — switching back to a tab shows previously loaded content without re-fetching. **Bounded to last 5 tabs** to prevent memory growth on mobile — when a 6th tab loads, the oldest cached tab's content is cleared (will re-fetch on next activation). Only the first page is cached; accumulated infinite scroll content is discarded on eviction. -- Each tab panel shows loading spinner, error state, retry button (re-fetches without full page reload), and empty state (same states as the old deck column) -- **AbortController:** Each tab's loading state includes an AbortController. When switching away from a loading tab, the in-flight client-side fetch is aborted. When switching back, if content wasn't loaded (cache miss), a fresh request starts. This prevents abandoned HTTP connections from piling up (especially important for hashtag tabs in Task 6). -- Full-width layout — no cramped columns, content fills the available width - -**Definition of Done:** - -- [ ] Clicking an Instance tab loads the first page of posts from the remote instance -- [ ] Posts display in full-width layout using `ap-item-card.njk` partial -- [ ] Infinite scroll loads more posts when scrolling near the bottom -- [ ] Loading spinner shown during initial load -- [ ] Error state with retry button shown on fetch failure (retry re-fetches without full page reload) -- [ ] Empty state shown when no posts available -- [ ] Switching away and back to a tab preserves already-loaded content (bounded to last 5 tabs) -- [ ] Switching away from a loading tab aborts the in-flight fetch (AbortController) - -**Verify:** - -- Pin `mastodon.social` (local) as a tab -- Click the tab — posts load in full-width layout -- Scroll down — more posts load via infinite scroll -- Switch to Search tab and back — previously loaded posts still visible - ---- - -### Task 6: Hashtag Tab Panel — Cross-Instance Aggregation - -**Objective:** When a Hashtag tab is active, query the hashtag timeline from pinned instance tabs in parallel (capped at 10), merge results by date, and deduplicate by post URL. Uses per-instance cursor pagination for correct multi-source paging. - -**Dependencies:** Task 2, Task 5 - -**Files:** - -- Create: `lib/controllers/hashtag-explore.js` — New API endpoint `hashtagExploreApiController` that takes a hashtag and per-instance cursor map, queries pinned instances in parallel, merges, deduplicates, and paginates -- Modify: `index.js` — Register the new hashtag explore API route -- Modify: `assets/reader-tabs.js` — Add hashtag tab loading logic (different API endpoint than instance tabs); manages per-instance cursor state client-side -- Modify: `views/activitypub-explore.njk` — Add hashtag tab panel template section with source instances info line - -**Key Decisions / Notes:** - -- `GET /admin/reader/api/explore/hashtag?hashtag={tag}&cursors={json}` — New endpoint -- **`MAX_HASHTAG_INSTANCES = 10`**: Hard cap on the number of instances queried per hashtag request. Queries the first 10 instance tabs by `order`. If more exist, the response includes `{ instancesQueried: 10, instancesTotal: N }` so the UI can show "Searching 10 of N instances". -- **Hashtag validation:** Validate hashtag via `validateHashtag()` from `tabs.js` before constructing remote API URLs. Reject invalid hashtags with 400. -- Reads instance tabs from `ap_explore_tabs` where `type === "instance"`, capped at 10 by `order`, then queries each instance's `/api/v1/timelines/tag/{hashtag}?local={scope}&limit=20` in parallel using `Promise.allSettled()` -- Results merged into a single array, sorted by `published` descending -- Deduplication by `uid` (post URL) — first occurrence wins (most recent fetch) -- **Per-instance cursor pagination:** The `cursors` query param is a JSON-encoded map of `{ domain: max_id }` pairs. On each request, each instance is queried with its own `max_id` from the cursor map. The response returns an updated cursor map reflecting the last item from each instance's results. The client stores this cursor map in Alpine state and sends it with the next "load more" request. This ensures correct pagination without missed or duplicate posts across instances with different timeline velocities. -- **Processing pipeline order:** (1) Fetch from all instances in parallel, (2) Merge by published date, (3) Dedup by URL, (4) Slice to page_size (20), (5) THEN render HTML via `request.app.render()` only for the returned items. This prevents wasting CPU rendering items that will be discarded. -- **Per-instance status in response metadata:** Response includes `sources` map: `{ "mastodon.social": "ok", "pixelfed.social": "error:404" }`. The hashtag tab panel shows a line like "Searching #indieweb across 3 instances: mastodon.social, fosstodon.org, ..." and "3 of 5 instances responded" when some fail. This makes the implicit coupling between instance tabs and hashtag tabs explicit. -- Timeout per instance: 10s (same as existing `FETCH_TIMEOUT_MS`). Failed instances excluded from results but reported in `sources`. -- If no instance tabs exist, returns empty results with a message "Pin some instances first" - -**Definition of Done:** - -- [ ] `GET /admin/reader/api/explore/hashtag?hashtag=indieweb` returns posts from pinned instances (up to 10) -- [ ] Hashtag validated via `validateHashtag()`, invalid hashtags return 400 -- [ ] Results sorted by published date descending -- [ ] Duplicate posts (same URL from multiple instances) deduplicated -- [ ] Per-instance status returned in response metadata (`sources` map) -- [ ] Hashtag tab panel shows "Searching #tag across N instances: domain1, domain2, ..." -- [ ] Infinite scroll works with per-instance cursor map pagination (no duplicates or gaps between pages) -- [ ] Maximum 10 instances queried per request (cap enforced) -- [ ] HTML rendering happens AFTER merge/dedup/paginate (not before) -- [ ] Empty state shown when no instance tabs exist (message: "Pin some instances first") -- [ ] Hashtag tab panel displays full-width timeline - -**Verify:** - -- Pin `mastodon.social` (local) and `fosstodon.org` (local) as instance tabs -- Add a `#indieweb` hashtag tab -- Click the hashtag tab — results from both instances appear, sorted by date -- Source line shows "Searching #indieweb across 2 instances: mastodon.social, fosstodon.org" -- No duplicate posts visible -- Scroll down — infinite scroll loads more posts without duplicates (per-instance cursors work correctly) -- Invalid hashtag (e.g., `../../path`) is rejected with 400 - ---- - -### Task 7: Cleanup — Remove Old Deck Code, Update CSS, Update Locales - -**Objective:** Remove all remaining references to the old deck system. Clean up CSS (remove `.ap-deck-*` classes), update locale strings, delete `assets/reader-decks.js` (note: `lib/controllers/decks.js` was already deleted in Task 2). - -**Dependencies:** Task 4, Task 5, Task 6 - -**Files:** - -- Delete: `assets/reader-decks.js` — Old Alpine deck components (fully replaced by `reader-tabs.js`) -- Modify: `assets/reader.css` — Remove all `.ap-deck-*` and `.ap-explore-deck-toggle*` styles (lines 2063-2248) -- Modify: `assets/reader-infinite-scroll.js` — Remove `apExploreScroll` component (replaced by tab-scoped scroll in `reader-tabs.js`); keep `apInfiniteScroll` unchanged -- Modify: `locales/en.json` — Remove `explore.deck.*` and `explore.tabs.decks` strings; ensure new tab strings are present -- Modify: `index.js` — Verify no remaining imports or references to `decks.js` - -**Key Decisions / Notes:** - -- `lib/controllers/decks.js` was already deleted in Task 2 — verify it's gone here -- `reader-infinite-scroll.js` still contains `apInfiniteScroll` for the main reader timeline — only remove `apExploreScroll` -- CSS cleanup: remove lines 2063-2248 from `reader.css` (deck toggle, deck grid, deck column, deck empty, deck responsive). Keep `.ap-tabs` styles (extended in Task 4). -- Locale cleanup: remove `explore.deck.*` object entirely, remove `explore.tabs.decks` string -- Verify `reader.css` does not exceed 300 lines per section after changes -- The `ap_decks` collection is left in MongoDB (not explicitly dropped). Users can manually drop it via `mongosh` if desired: `db.ap_decks.drop()` - -**Definition of Done:** - -- [ ] `lib/controllers/decks.js` confirmed deleted (was done in Task 2) -- [ ] `assets/reader-decks.js` deleted -- [ ] No `.ap-deck-*` CSS classes remain in `reader.css` -- [ ] `apExploreScroll` retained in `reader-infinite-scroll.js` (still used by Search tab's server-rendered infinite scroll) -- [ ] `apInfiniteScroll` still works in `reader-infinite-scroll.js` -- [ ] No `deck` or `ap_decks` references remain anywhere in codebase (except git history) -- [ ] All locale strings clean — no orphaned deck strings - -**Verify:** - -- `grep -r "ap_decks\|apDeckColumn\|apDeckToggle\|reader-decks" --include="*.js" --include="*.njk" --include="*.json" lib/ views/ assets/ locales/ index.js` returns nothing (apExploreScroll intentionally retained for Search tab) -- `grep -r "ap-deck-" assets/reader.css` returns nothing - -## Testing Strategy - -- **Unit tests:** No automated test suite exists for this plugin (manual testing only — see CLAUDE.md). However, each task will be verified by: - 1. Checking that the explore page renders correctly via Playwright - 2. Testing API endpoints with curl - 3. Verifying infinite scroll works -- **Integration tests:** Test the full tab lifecycle: add instance tab → browse timeline → add hashtag tab → verify aggregation → reorder → remove -- **Manual verification:** - 1. `playwright-cli open https://rmendes.net/activitypub/admin/reader/explore` — verify UI renders - 2. `curl` the tab API endpoints to verify CRUD operations - 3. Test with multiple instances to verify hashtag aggregation - 4. Test responsive layout on mobile widths - -## Risks and Mitigations - -| Risk | Likelihood | Impact | Mitigation | -| --- | --- | --- | --- | -| Hashtag aggregation slow with many instances | Med | Med | Hard cap at MAX_HASHTAG_INSTANCES = 10; `Promise.allSettled()` with per-instance 10s timeout; exclude failed instances; show partial results with source status | -| Hashtag input injection / path traversal | Med | High | `validateHashtag()` enforces `/^[\w]+$/` regex, max 100 chars, strips leading `#`. Called in both tab CRUD and hashtag explore endpoint | -| Mastodon API rate limiting on hashtag queries | Low | Med | Each tab loads independently on user click, not all at once on page load; 10s timeout per instance prevents hanging | -| Tab reordering race condition (concurrent clicks) | Low | Low | Client-side debouncing (500ms) batches rapid arrow clicks into single API call; reorder endpoint accepts full ordered array | -| MongoDB unique index bypass with null fields | Med | Med | All insertions explicitly set ALL four indexed fields (unused fields set to `null`); documented in Task 1 and Task 2 | -| Abandoned HTTP connections on tab switch | Low | Med | AbortController aborts in-flight client fetch when switching away from a loading tab | -| Old `ap_decks` data remains in MongoDB | Low | Low | Old collection is simply not registered; data stays in MongoDB but is unused. User can manually drop via `mongosh` if desired | -| CSS file exceeds 300 line threshold after changes | Low | Med | Deck CSS removal (~185 lines) roughly offsets new tab CSS addition (~100 lines); net reduction in CSS | - -## Open Questions - -- None — all design decisions were made during brainstorming and refined by plan review findings. - -### Deferred Ideas - -- Drag-and-drop tab reordering (enhancement over up/down arrows) -- Per-instance hashtag filter within instance tabs -- Auto-refresh / live polling for active tabs -- Tab color customization -- Short-TTL caching (30-60s) for hashtag aggregation results to reduce re-querying on rapid scroll diff --git a/docs/plans/2026-03-02-reader-timeline-enhancements.md b/docs/plans/2026-03-02-reader-timeline-enhancements.md deleted file mode 100644 index 3557f07..0000000 --- a/docs/plans/2026-03-02-reader-timeline-enhancements.md +++ /dev/null @@ -1,40 +0,0 @@ -# Reader Timeline Enhancements - -## Features - -### 1. New Posts Detection (Reader timeline) - -30-second background poll checks for items newer than the top-most visible item's `published` date. - -- **API**: `GET /admin/reader/api/timeline/count-new?after={isoDate}&tab={tab}` returns `{ count: N }` -- **UI**: Sticky banner at top of timeline: "N new posts — Load" -- Clicking loads new items via existing `api/timeline` with `after=` param, prepends to timeline -- Banner disappears after loading; polling continues from newest item's date -- Explore tabs excluded (external instance APIs don't support "since" queries efficiently) - -### 2. Mark As Read on Scroll - -IntersectionObserver watches each `.ap-card` at 50% threshold. - -- When card is 50% visible, its `uid` is batched client-side -- Every 5 seconds, batch flushes via `POST /admin/reader/api/timeline/mark-read` with `{ uids: [...] }` -- Server sets `{ read: true }` on matching `ap_timeline` docs -- **Visual**: `.ap-card--read` class applies `opacity: 0.7`, set immediately on observe -- **Filter toggle**: "Show unread only" in tab bar adds `?unread=1` — server filters `{ read: { $ne: true } }` -- `unreadCount` in template reflects actual unread items - -### 3. Infinite Scroll + Load More - -Already implemented via `apInfiniteScroll` and `apExploreScroll` Alpine components. No changes needed. - -## Files to Modify - -| File | Change | -|------|--------| -| `lib/controllers/api-timeline.js` | New `countNewController` and `markReadController` endpoints | -| `lib/storage/timeline.js` | `countNewItems()` and `markItemsRead()` functions | -| `lib/controllers/reader.js` | Pass `unread` filter param, compute `unreadCount` from DB | -| `index.js` | Register new API routes | -| `assets/reader-infinite-scroll.js` | New `apNewPostsBanner` Alpine component + read tracking observer | -| `views/activitypub-reader.njk` | New posts banner markup, unread toggle, read class on cards | -| `assets/reader.css` | `.ap-card--read`, banner styles, unread toggle styles | diff --git a/docs/plans/2026-03-03-reader-improvements-plan.md b/docs/plans/2026-03-03-reader-improvements-plan.md deleted file mode 100644 index 8a9028b..0000000 --- a/docs/plans/2026-03-03-reader-improvements-plan.md +++ /dev/null @@ -1,996 +0,0 @@ -# Reader Improvements Plan — Inspired by Elk & Phanpy - -**Date:** 2026-03-03 -**Source:** `docs/research/2026-03-03-elk-phanpy-comparison.md` -**Current version:** 2.4.5 - ---- - -## Overview - -Prioritized improvements to the ActivityPub reader, organized into releases. Each release is a publishable npm version. Tasks within a release are ordered by dependency (later tasks may depend on earlier ones). - -**Release 0 must ship first** — it unifies the reader and explore pipelines so that every subsequent release only needs to implement each feature once. - ---- - -## Release 0: Unify Reader & Explore Pipeline (v2.5.0-rc.1) - -**Impact:** Critical prerequisite — Without this, every improvement from Releases 1-8 must be implemented twice (once for inbox-sourced items, once for Mastodon API items), with different code in different files. This release eliminates that duplication. - -### Problem Statement - -The reader (followed accounts) and explore (public instance timelines) are the same feature with different data sources. But the code treats them as separate systems: - -| Operation | Reader | Explore | Duplicated? | -|-----------|--------|---------|-------------| -| Item construction | `extractObjectData()` in `timeline-store.js` | `mapMastodonStatusToItem()` in `explore-utils.js` | Yes — same shape, different source | -| Quote stripping | `reader.js:200-206` | `explore.js:102-108` AND `explore.js:193-199` | Yes — identical loop in 3 places | -| Moderation filtering | `reader.js:84-146` | (missing) | Explore has none | -| Interaction map | `reader.js:154-198` | `explore.js:134` (empty `{}`) | Different but same pattern | -| Tab filtering | `reader.js:59-82` | N/A | Reader-only | -| Mastodon API fetch | N/A | `explore.js:63-114` AND `explore.js:160-205` | Duplicated within explore itself | -| Card HTML rendering | `api-timeline.js:148-170` | `explore.js:207-229` | Identical | -| Infinite scroll JS | `apInfiniteScroll` (95 lines) | `apExploreScroll` (93 lines) | 80% identical | - -Additionally, `reader.js` and `api-timeline.js` duplicate the same logic (moderation, interaction map, tab filtering, quote stripping) — the API endpoint is a copy-paste of the page controller. - -### Task 0.1: Extract `postProcessItems()` shared utility - -**File:** `lib/item-processing.js` (new) - -Extract the shared post-processing that happens after items are loaded (from DB or API), regardless of source. This function takes raw items and returns processed items ready for rendering. - -```js -/** - * Post-process timeline items for rendering. - * Used by both reader and explore controllers. - * - * @param {Array} items - Raw timeline items (from DB or Mastodon API mapping) - * @param {object} options - * @param {object} [options.moderation] - { mutedUrls, mutedKeywords, blockedUrls, filterMode } - * @param {object} [options.interactionsCol] - MongoDB collection for interaction state lookup - * @returns {{ items: Array, interactionMap: object }} - */ -export async function postProcessItems(items, options = {}) { - // 1. Apply moderation filters (muted actors, keywords, blocked actors) - if (options.moderation) { - items = applyModerationFilters(items, options.moderation); - } - - // 2. Strip "RE:" paragraphs from items with quote embeds - stripQuoteReferences(items); - - // 3. Build interaction map (likes, boosts) — empty for explore - const interactionMap = options.interactionsCol - ? await buildInteractionMap(items, options.interactionsCol) - : {}; - - return { items, interactionMap }; -} -``` - -This eliminates 4 copies of the quote-stripping loop, 2 copies of the moderation filter, and 2 copies of the interaction map builder. - -### Task 0.2: Extract `applyModerationFilters()` into shared utility - -**File:** `lib/item-processing.js` - -Move the moderation filtering logic from `reader.js:84-146` (and its duplicate in `api-timeline.js:63-111`) into a single function: - -```js -export function applyModerationFilters(items, { mutedUrls, mutedKeywords, blockedUrls, filterMode }) { - const blockedSet = new Set(blockedUrls); - const mutedSet = new Set(mutedUrls); - - if (blockedSet.size === 0 && mutedSet.size === 0 && mutedKeywords.length === 0) { - return items; - } - - return items.filter((item) => { - if (item.author?.url && blockedSet.has(item.author.url)) return false; - // ... (existing logic, written once) - }); -} -``` - -### Task 0.3: Extract `buildInteractionMap()` into shared utility - -**File:** `lib/item-processing.js` - -Move the interaction map logic from `reader.js:154-198` (and `api-timeline.js:113-136`) into: - -```js -export async function buildInteractionMap(items, interactionsCol) { - const lookupUrls = new Set(); - const objectUrlToUid = new Map(); - for (const item of items) { /* ... existing logic ... */ } - // Returns { [uid]: { like: true, boost: true } } -} -``` - -### Task 0.4: Extract `renderItemCards()` shared HTML renderer - -**File:** `lib/item-processing.js` - -Move the server-side card rendering from `api-timeline.js:148-170` (and identical code in `explore.js:207-229`) into: - -```js -/** - * Render items to HTML using ap-item-card.njk. - * Used by both timeline API and explore API for infinite scroll. - */ -export async function renderItemCards(items, request, templateData) { - const htmlParts = await Promise.all( - items.map((item) => new Promise((resolve, reject) => { - request.app.render( - "partials/ap-item-card.njk", - { ...templateData, item }, - (err, html) => err ? reject(err) : resolve(html), - ); - })), - ); - return htmlParts.join(""); -} -``` - -### Task 0.5: Deduplicate Mastodon API fetch in explore controller - -**File:** `lib/controllers/explore.js` - -`exploreController()` (page load) and `exploreApiController()` (AJAX scroll) have 95% identical fetch logic. Extract: - -```js -/** - * Fetch statuses from a remote Mastodon-compatible instance. - * @returns {{ items: Array, nextMaxId: string|null }} - */ -async function fetchMastodonTimeline(instance, { scope, hashtag, maxId, limit }) { - const isLocal = scope === "local"; - let apiUrl; - if (hashtag) { - apiUrl = new URL(`https://${instance}/api/v1/timelines/tag/${encodeURIComponent(hashtag)}`); - } else { - apiUrl = new URL(`https://${instance}/api/v1/timelines/public`); - } - apiUrl.searchParams.set("local", isLocal ? "true" : "false"); - apiUrl.searchParams.set("limit", String(limit)); - if (maxId) apiUrl.searchParams.set("max_id", maxId); - - const controller = new AbortController(); - const timeoutId = setTimeout(() => controller.abort(), FETCH_TIMEOUT_MS); - const fetchRes = await fetch(apiUrl.toString(), { - headers: { Accept: "application/json" }, - signal: controller.signal, - }); - clearTimeout(timeoutId); - - if (!fetchRes.ok) throw new Error(`Remote returned HTTP ${fetchRes.status}`); - const statuses = await fetchRes.json(); - if (!Array.isArray(statuses)) throw new Error("Unexpected API response"); - - const items = statuses.map((s) => mapMastodonStatusToItem(s, instance)); - const nextMaxId = (statuses.length === limit && statuses.length > 0) - ? statuses[statuses.length - 1].id - : null; - - return { items, nextMaxId }; -} -``` - -Both controllers call this instead of duplicating the fetch. - -### Task 0.6: Simplify reader controller and API controller - -**Files:** `lib/controllers/reader.js`, `lib/controllers/api-timeline.js` - -Rewrite both to use `postProcessItems()`: - -**reader.js** (before — 70 lines of processing): -```js -const result = await getTimelineItems(collections, options); -let items = applyTabFilter(result.items, tab); - -const moderation = await loadModerationData(modCollections); -const { items: processed, interactionMap } = await postProcessItems(items, { - moderation, - interactionsCol: application?.collections?.get("ap_interactions"), -}); -``` - -**api-timeline.js** (before — 100 lines of duplicated processing): -```js -const result = await getTimelineItems(collections, options); -let items = applyTabFilter(result.items, tab); - -const moderation = await loadModerationData(modCollections); -const { items: processed, interactionMap } = await postProcessItems(items, { - moderation, - interactionsCol: application?.collections?.get("ap_interactions"), -}); -const html = await renderItemCards(processed, request, { ...response.locals, mountPath, csrfToken, interactionMap }); -response.json({ html, before: result.before }); -``` - -### Task 0.7: Simplify explore controllers - -**File:** `lib/controllers/explore.js` - -Rewrite both `exploreController()` and `exploreApiController()` to use `fetchMastodonTimeline()`, `postProcessItems()`, and `renderItemCards()`: - -```js -export function exploreApiController(mountPath) { - return async (request, response, next) => { - const instance = validateInstance(request.query.instance); - if (!instance) return response.status(400).json({ error: "Invalid instance" }); - - const { items, nextMaxId } = await fetchMastodonTimeline(instance, { - scope: request.query.scope, - hashtag: validateHashtag(request.query.hashtag), - maxId: request.query.max_id, - limit: MAX_RESULTS, - }); - - const { items: processed, interactionMap } = await postProcessItems(items); - const html = await renderItemCards(processed, request, { - ...response.locals, mountPath, csrfToken: getToken(request.session), interactionMap, - }); - - response.json({ html, maxId: nextMaxId }); - }; -} -``` - -### Task 0.8: Extract `applyTabFilter()` shared utility - -**File:** `lib/item-processing.js` - -The tab filtering logic is duplicated between `reader.js:71-82` and `api-timeline.js:49-61`: - -```js -export function applyTabFilter(items, tab) { - if (tab === "replies") return items.filter((item) => item.inReplyTo); - if (tab === "media") return items.filter((item) => - item.photo?.length > 0 || item.video?.length > 0 || item.audio?.length > 0 - ); - return items; -} -``` - -### Task 0.9: Unify infinite scroll Alpine component - -**File:** `assets/reader-infinite-scroll.js` - -Replace `apExploreScroll` and `apInfiniteScroll` with a single parameterized `apInfiniteScroll` component: - -```js -Alpine.data("apInfiniteScroll", () => ({ - loading: false, - done: false, - cursor: null, // Generic cursor — was "maxId" for explore, "before" for reader - apiUrl: "", // Set from data-api-url attribute - cursorParam: "", // Set from data-cursor-param ("max_id" or "before") - cursorField: "", // Response field name for next cursor ("maxId" or "before") - extraParams: {}, // Additional query params (instance, scope, hashtag, tab, tag) - observer: null, - - init() { - const el = this.$el; - this.cursor = el.dataset.cursor || null; - this.apiUrl = el.dataset.apiUrl || ""; - this.cursorParam = el.dataset.cursorParam || "before"; - this.cursorField = el.dataset.cursorField || "before"; - - // Parse extra params from data-extra-params JSON attribute - try { - this.extraParams = JSON.parse(el.dataset.extraParams || "{}"); - } catch { this.extraParams = {}; } - - if (!this.cursor) { this.done = true; return; } - - this.observer = new IntersectionObserver( - (entries) => { - for (const entry of entries) { - if (entry.isIntersecting && !this.loading && !this.done) { - this.loadMore(); - } - } - }, - { rootMargin: "200px" }, - ); - - if (this.$refs.sentinel) this.observer.observe(this.$refs.sentinel); - }, - - async loadMore() { - if (this.loading || this.done || !this.cursor) return; - this.loading = true; - - const params = new URLSearchParams({ - [this.cursorParam]: this.cursor, - ...this.extraParams, - }); - - try { - const res = await fetch(`${this.apiUrl}?${params}`, { - headers: { Accept: "application/json" }, - }); - if (!res.ok) throw new Error(`HTTP ${res.status}`); - const data = await res.json(); - - const timeline = this.$refs.timeline || this.$el.querySelector("[data-timeline]"); - if (data.html && timeline) { - timeline.insertAdjacentHTML("beforeend", data.html); - } - - if (data[this.cursorField]) { - this.cursor = data[this.cursorField]; - } else { - this.done = true; - if (this.observer) this.observer.disconnect(); - } - } catch (err) { - console.error("[ap-infinite-scroll] load failed:", err.message); - } finally { - this.loading = false; - } - }, - - destroy() { - if (this.observer) this.observer.disconnect(); - }, -})); -``` - -Template usage for reader: -```njk -
-``` - -Template usage for explore: -```njk -
-``` - -### Task 0.10: Update templates to use unified component - -**Files:** `views/activitypub-reader.njk`, `views/activitypub-explore.njk` - -Replace `x-data="apExploreScroll"` with `x-data="apInfiniteScroll"` using the parameterized data attributes. Remove the `apExploreScroll` component definition. - -### Task 0.11: Verify no regressions - -Manual testing: -- Reader timeline loads, infinite scroll works, new posts banner works -- Explore search tab loads, infinite scroll works -- Explore pinned tabs load, load-more buttons work -- Quote embeds render in both views -- Moderation filtering still works in reader -- Interaction state (likes/boosts) still shows in reader -- Read tracking still works - -### Files changed - -| File | Change | -|------|--------| -| `lib/item-processing.js` | **New** — `postProcessItems()`, `applyModerationFilters()`, `buildInteractionMap()`, `renderItemCards()`, `applyTabFilter()`, `stripQuoteReferences()` | -| `lib/controllers/reader.js` | Simplified — uses `postProcessItems()` | -| `lib/controllers/api-timeline.js` | Simplified — uses `postProcessItems()` + `renderItemCards()` | -| `lib/controllers/explore.js` | Simplified — uses `fetchMastodonTimeline()`, `postProcessItems()`, `renderItemCards()` | -| `assets/reader-infinite-scroll.js` | Unified — single `apInfiniteScroll` component replaces two | -| `views/activitypub-reader.njk` | Updated data attributes for unified scroll component | -| `views/activitypub-explore.njk` | Updated data attributes for unified scroll component | - -### Impact on subsequent releases - -After Release 0, every improvement only needs to be added in ONE place: - -| Enhancement | Before Release 0 | After Release 0 | -|-------------|-------------------|-----------------| -| Custom emoji | `timeline-store.js` + `explore-utils.js` + `reader.js` + `explore.js` | `item-processing.js` (single post-process step) | -| Quote stripping | 4 locations | `item-processing.js` only | -| Moderation | 2 locations | `item-processing.js` only | -| New content transforms | Must add to both pipelines | Single pipeline | - ---- - -## Release 1: Custom Emoji Rendering (v2.5.0) - -**Impact:** High — Custom emoji is ubiquitous on the fediverse. Without it, display names show raw `:shortcode:` text and post content loses visual meaning. - -### Task 1.1: Store emoji data from ActivityPub inbox - -**File:** `lib/timeline-store.js` - -`extractObjectData()` currently ignores emoji data. Fedify's `Note`/`Article` objects expose custom emoji via the `getTags()` call — emoji are `Emoji` instances (a subclass of `Flag`) in the tags array, alongside `Hashtag` and `Mention`. - -**Changes:** -- In the tag extraction loop (~line 190), check for Fedify `Emoji` instances -- Each Emoji has: `name` (`:shortcode:` with colons), and an `icon` property (an `Image` with `url`) -- Extract to an `emojis` array: `[{ shortcode: "blobcat", url: "https://..." }]` -- Add `emojis` to the returned item object -- Also extract emojis from the actor object in `extractActorInfo()` for display name emoji - -**Stored data shape:** -```js -emojis: [ - { shortcode: "blobcat", url: "https://cdn.example/emoji/blobcat.png" }, - { shortcode: "verified", url: "https://cdn.example/emoji/verified.png" } -] -``` - -### Task 1.2: Store emoji data from Mastodon REST API (explore view) - -**File:** `lib/controllers/explore-utils.js` - -Mastodon's REST API v1 returns `status.emojis` as an array of `{ shortcode, url, static_url, visible_in_picker }` objects, and `status.account.emojis` for display name emoji. - -**Changes:** -- In `mapMastodonStatusToItem()`, extract `status.emojis` → `item.emojis` -- Extract `account.emojis` → `item.author.emojis` -- Normalize to same shape as Task 1.1: `[{ shortcode, url }]` - -### Task 1.3: Create emoji replacement utility - -**File:** `lib/emoji-utils.js` (new) - -A small utility that replaces `:shortcode:` patterns with `` tags. Used in both content HTML and display names. - -```js -export function replaceCustomEmoji(html, emojis) { - if (!emojis?.length) return html; - for (const emoji of emojis) { - const pattern = new RegExp(`:${escapeRegex(emoji.shortcode)}:`, "g"); - html = html.replace(pattern, - `:${emoji.shortcode}:` - ); - } - return html; -} -``` - -Must escape regex special characters in shortcodes. Must be called AFTER `sanitizeContent()` (which would strip the `` tags if run after). - -### Task 1.4: Apply emoji replacement in content pipeline - -**File:** `lib/item-processing.js` - -Add an `applyCustomEmoji(items)` step to `postProcessItems()`. Since both reader and explore flow through this single function (after Release 0), emoji replacement happens once for all items regardless of source. - -```js -// Inside postProcessItems(), after quote stripping: -applyCustomEmoji(items); -``` - -The function iterates items, calling `replaceCustomEmoji(item.content.html, item.emojis)` on each. - -### Task 1.5: Apply emoji replacement in display names - -**File:** `lib/item-processing.js` - -Add emoji replacement for display names inside the same `applyCustomEmoji()` step: - -```js -if (item.author?.emojis?.length && item.author.name) { - item.author.nameHtml = replaceCustomEmoji( - sanitizeHtml(item.author.name, { allowedTags: [], allowedAttributes: {} }), - item.author.emojis, - ); -} -``` - -This adds `author.nameHtml` alongside existing `author.name`. Template renders `nameHtml | safe` when present, falls back to `name`. - -### Task 1.6: Add emoji CSS - -**File:** `assets/reader.css` - -```css -.ap-custom-emoji { - height: 1.2em; - width: auto; - vertical-align: middle; - display: inline; - margin: 0 0.05em; -} -``` - -### Task 1.7: Update sanitize-html allowlist - -**File:** `lib/timeline-store.js` (or wherever `sanitizeContent` config lives) - -The `sanitize-html` configuration must allow `` tags with class `ap-custom-emoji` through — but only for emoji images, not arbitrary remote images. Since emoji replacement happens AFTER sanitization, this isn't an issue: the emoji `` tags are inserted post-sanitization and never pass through the sanitizer. - -Verify this ordering is correct in both codepaths (inbox + explore). - -### Task 1.8: Store emoji in MongoDB - -**File:** `lib/storage/timeline.js` - -Add `emojis` to the stored fields in `addTimelineItem()`. Also add `author.emojis` if storing per-author emoji data. - ---- - -## Release 2: Relative Timestamps (v2.5.1) - -**Impact:** High — Every fediverse client shows "2m ago" instead of "Feb 25, 2026, 4:46 PM". Relative timestamps are dramatically faster to scan when reading a timeline. - -### Task 2.1: Create relative time Alpine directive - -**File:** `assets/reader-relative-time.js` (new) - -A small Alpine.js directive that: -1. Reads `datetime` attribute from a `
` that closes `.ap-item__content`) and before attachments/link preview, add: - -```nunjucks - {# Poll options #} - {% if item.type == "question" or (item.pollOptions and item.pollOptions.length > 0) %} - {% include "partials/ap-poll-options.njk" %} - {% endif %} -``` - -### Step 7: Add CSS for poll rendering - -In `assets/reader.css`, add a new section: - -```css -/* ========================================================================== - Poll / Question - ========================================================================== */ - -.ap-poll { - margin-top: var(--space-s); -} - -.ap-poll__option { - position: relative; - padding: var(--space-xs) var(--space-s); - margin-bottom: var(--space-xs); - border-radius: var(--border-radius-small); - background: var(--color-offset); - overflow: hidden; -} - -.ap-poll__bar { - position: absolute; - top: 0; - left: 0; - bottom: 0; - background: var(--color-primary); - opacity: 0.15; - border-radius: var(--border-radius-small); -} - -.ap-poll__label { - position: relative; - font-size: var(--font-size-s); - color: var(--color-on-background); -} - -.ap-poll__votes { - position: relative; - float: right; - font-size: var(--font-size-s); - font-weight: 600; - color: var(--color-on-offset); -} - -.ap-poll__footer { - font-size: var(--font-size-xs); - color: var(--color-on-offset); - margin-top: var(--space-xs); -} -``` - -### Step 8: Add i18n strings - -In `locales/en.json`, add: - -```json -"poll": { - "voters": "voters", - "votes": "votes", - "closed": "Poll closed", - "endsAt": "Ends" -} -``` - -### Step 9: Verify syntax - -Run: `node -c lib/timeline-store.js` -Expected: No errors - -### Step 10: Commit - -```bash -git add lib/timeline-store.js views/partials/ap-poll-options.njk views/partials/ap-item-card.njk assets/reader.css locales/en.json -git commit -m "feat: inbound poll/question support — parse and render vote options" -``` - ---- - -## Task 5: Flag Handler (Inbound Reports) - -Other fediverse servers can send `Flag` activities to report abusive content or actors. Currently these are silently dropped. This task adds a `Flag` inbox listener, an `ap_reports` collection, admin notification, and a reports view in the moderation dashboard. - -**Files:** -- Modify: `lib/inbox-listeners.js` (add Flag handler) -- Modify: `index.js` (register ap_reports collection + indexes) -- Modify: `lib/storage/notifications.js:129` (add "report" to type counts) -- Modify: `views/partials/ap-notification-card.njk` (add report notification type) -- Modify: `views/activitypub-notifications.njk` (add Reports tab) -- Modify: `lib/controllers/reader.js` (add "report" to validTabs) -- Modify: `locales/en.json` (add report i18n strings) - -### Step 1: Register `ap_reports` collection in `index.js` - -In the collection registration block (around line 891), add: - -```javascript -Indiekit.addCollection("ap_reports"); -``` - -In the collection storage block (around line 910), add: - -```javascript -ap_reports: indiekitCollections.get("ap_reports"), -``` - -In the indexes block (around line 1000), add: - -```javascript -// ap_reports indexes -try { - await this._collections.ap_reports.createIndex( - { createdAt: 1 }, - { expireAfterSeconds: notifRetention || undefined }, - ); - await this._collections.ap_reports.createIndex({ reporterUrl: 1 }); - await this._collections.ap_reports.createIndex({ reportedUrl: 1 }); -} catch { - // Indexes may already exist -} -``` - -### Step 2: Add Flag inbox listener - -In `lib/inbox-listeners.js`, add the `Flag` import to the destructure from `@fedify/fedify/vocab` (around line 6-24). Then add a new handler after the Block handler (after line ~744): - -```javascript - // ── Flag (Report) ────────────────────────────────────────────── - .on(Flag, async (ctx, flag) => { - try { - const authLoader = getAuthLoader ? await getAuthLoader(ctx) : undefined; - const actorObj = await flag.getActor({ documentLoader: authLoader }).catch(() => null); - - const reporterUrl = actorObj?.id?.href || flag.actorId?.href || ""; - const reporterName = actorObj?.name?.toString() || actorObj?.preferredUsername?.toString() || reporterUrl; - - // Extract reported objects — Flag can report actors or posts - const reportedIds = flag.objectIds?.map((u) => u.href) || []; - const reason = flag.content?.toString() || ""; - - if (reportedIds.length === 0 && !reason) { - console.info("[ActivityPub] Ignoring empty Flag from", reporterUrl); - return; - } - - // Store report - if (collections.ap_reports) { - await collections.ap_reports.insertOne({ - reporterUrl, - reporterName, - reportedUrls: reportedIds, - reason, - createdAt: new Date().toISOString(), - read: false, - }); - } - - // Create notification - if (collections.ap_notifications) { - const { addNotification } = await import("./storage/notifications.js"); - await addNotification(collections, { - uid: `flag:${reporterUrl}:${Date.now()}`, - type: "report", - actorUrl: reporterUrl, - actorName: reporterName, - actorPhoto: actorObj?.iconUrl?.href || actorObj?.icon?.url?.href || "", - actorHandle: actorObj?.preferredUsername - ? `@${actorObj.preferredUsername}@${new URL(reporterUrl).hostname}` - : reporterUrl, - objectUrl: reportedIds[0] || "", - summary: reason ? reason.slice(0, 200) : "Report received", - published: new Date().toISOString(), - }); - } - - await logActivity(collections, { - direction: "inbound", - type: "Flag", - actorUrl: reporterUrl, - objectUrl: reportedIds[0] || "", - summary: `Report from ${reporterName}: ${reason.slice(0, 100)}`, - }); - - console.info(`[ActivityPub] Flag received from ${reporterName} — ${reportedIds.length} objects reported`); - } catch (error) { - console.warn("[ActivityPub] Flag handler error:", error.message); - } - }) -``` - -### Step 3: Update notification type handling - -In `lib/storage/notifications.js`, in `getNotificationCountsByType()` (around line 129), add `report: 0` to the counts object: - -```javascript -const counts = { all: 0, reply: 0, like: 0, boost: 0, follow: 0, dm: 0, report: 0 }; -``` - -And add the case to handle `_id === "report"`. - -### Step 4: Update notification card template - -In `views/partials/ap-notification-card.njk`, add the report type badge and action text: - -Type badge (alongside other `elif` checks): -```nunjucks -{% elif item.type == "report" %}⚑ -``` - -Action text: -```nunjucks -{% elif item.type == "report" %}{{ __("activitypub.reports.sentReport") }} -``` - -### Step 5: Add Reports tab to notifications - -In `views/activitypub-notifications.njk`, add a Reports tab alongside the DMs tab: - -```nunjucks - - {{ __("activitypub.notifications.tabs.reports") }} - {% if counts.report > 0 %} - {{ counts.report }} - {% endif %} - -``` - -### Step 6: Add "report" to valid tabs - -In `lib/controllers/reader.js`, add `"report"` to the `validTabs` array: - -```javascript -const validTabs = ["all", "reply", "like", "boost", "follow", "dm", "report"]; -``` - -### Step 7: Add i18n strings - -In `locales/en.json`, add: - -```json -"reports": { - "sentReport": "filed a report", - "title": "Reports" -} -``` - -And add to `notifications.tabs`: - -```json -"reports": "Reports" -``` - -### Step 8: Verify syntax - -Run: `node -c lib/inbox-listeners.js && node -c lib/storage/notifications.js && node -c index.js` -Expected: No errors - -### Step 9: Commit - -```bash -git add lib/inbox-listeners.js lib/storage/notifications.js views/partials/ap-notification-card.njk views/activitypub-notifications.njk lib/controllers/reader.js locales/en.json index.js -git commit -m "feat: inbound Flag handler — receive and display abuse reports" -``` - ---- - -## Verification Plan - -After all tasks are implemented: - -1. **Outbound Delete** — Create a test post, syndicate to fediverse. Delete from Indiekit. Call `POST /activitypub/admin/federation/delete` with the post URL. Check activity log shows outbound Delete. Verify from a Mastodon account that the post is removed. - -2. **Visibility** — Set `defaultVisibility: "unlisted"` in config. Create a post. Check from Mastodon that the post appears in the home timeline of followers but NOT on the public/federated timeline. Reset to "public". - -3. **Content Warning** — Create a post with `sensitive: true` and a `summary` field via Micropub. Verify from Mastodon that the post shows behind a CW toggle with the summary text. - -4. **Polls** — From Mastodon, create a poll and post it. View the reader timeline. Verify poll options render with percentage bars and voter count. - -5. **Reports** — From a Mastodon instance, report the test actor. Check that: - - A notification appears in the Reports tab - - The activity log shows an inbound Flag - - The `ap_reports` collection has the report entry - ---- - -## Summary - -| Task | Gap Closed | Priority | Files Changed | -|------|-----------|----------|---------------| -| 1 | Outbound Delete | P1 — High | index.js, new controller, en.json | -| 2 | Unlisted + Followers-only | P1/P2 | jf2-to-as2.js, index.js | -| 3 | Content Warning (outbound) | P2 | jf2-to-as2.js | -| 4 | Question/Poll (inbound) | P2 | timeline-store.js, new partial, item-card, CSS, en.json | -| 5 | Flag Handler (inbound) | P2 | inbox-listeners.js, notifications.js, templates, en.json, index.js | -| — | Block (outbound) | **Already implemented** | No work needed | - -**Estimated coverage after implementation:** ~85% of Fedify capabilities, ~98% of real-world fediverse traffic. diff --git a/docs/research/2026-03-03-elk-phanpy-comparison.md b/docs/research/2026-03-03-elk-phanpy-comparison.md deleted file mode 100644 index 105d795..0000000 --- a/docs/research/2026-03-03-elk-phanpy-comparison.md +++ /dev/null @@ -1,440 +0,0 @@ -# Elk & Phanpy Deep Dive — Lessons for Our ActivityPub Reader - -**Date:** 2026-03-03 -**Purpose:** Identify concrete improvements by comparing our reader with two best-in-class fediverse clients. - ---- - -## Architecture Comparison - -| Aspect | Elk (Vue/Nuxt) | Phanpy (React/Vite) | Our Reader (Nunjucks/Alpine) | -|--------|----------------|---------------------|------------------------------| -| Rendering | Client-side SPA | Client-side SPA | Server-side HTML + Alpine sprinkles | -| Content processing | AST parse → VNode tree | DOM manipulation pipeline | Server-side sanitize-html | -| State management | Vue refs + composables | Valtio proxy state | Alpine.js `x-data` components | -| Pagination | Virtual scroller + stream | IntersectionObserver + debounce | IntersectionObserver + cursor | -| CSS | UnoCSS (Tailwind-like) | CSS Modules + custom properties | Indiekit theme custom properties | - -**Key insight:** Both Elk and Phanpy are full SPAs with rich client-side rendering. Our server-rendered approach is fundamentally different — we can't replicate everything, but we can cherry-pick the most impactful patterns. - ---- - -## 1. Content Rendering - -### What Elk & Phanpy Do Better - -**Elk's content pipeline:** -1. Parse HTML into AST (ultrahtml) -2. Sanitize with element whitelist -3. Transform mentions → interactive hover cards -4. Transform hashtags → hover cards with usage stats -5. Transform emoji shortcodes → inline images with tooltips -6. Transform code blocks (backtick syntax) -7. Render as Vue VNodes - -**Phanpy's content pipeline:** -1. Parse HTML into DOM -2. Shorten long URLs (>30 chars): `https://...example.com/long` -3. Detect hashtag stuffing (3+ tags in paragraph) → collapse -4. Replace custom emoji shortcodes with `` elements -5. Convert backtick code blocks to `
`
-6. Add `is-quote` class to quote links in content
-7. Wrap bare text in `` for Safari text-decoration fix
-
-### What We're Missing
-
-| Feature | Elk | Phanpy | Us | Priority |
-|---------|-----|--------|-----|----------|
-| Custom emoji rendering | ✅ `:emoji:` → `` with tooltip | ✅ `:emoji:` → `` | ❌ Raw shortcodes shown | **High** |
-| Long URL shortening | ❌ | ✅ Truncate >30 chars | ❌ Full URLs shown | Medium |
-| Hashtag stuffing collapse | ❌ | ✅ 3+ tags collapsed | ❌ All tags shown inline | Low |
-| Mention hover cards | ✅ Full profile card on hover | ❌ | ❌ Links only | Low (needs client JS) |
-| Code block rendering | ✅ Syntax highlighting | ✅ Backtick → `
` | ❌ Pass-through only | Low |
-| Inline code | ✅ | ✅ Backtick → `` | ❌ | Low |
-
-### Recommended Action: Custom Emoji
-
-Both clients treat this as essential. Implementation for server-rendered HTML:
-
-In `sanitizeContent()` or a new `processEmoji()` step, replace `:shortcode:` with `` tags using the emoji data from the Mastodon API status object (`status.emojis` array). Each emoji has `{ shortcode, url, static_url }`.
-
-```js
-// In timeline-store.js or explore-utils.js
-function replaceCustomEmoji(html, emojis) {
-  if (!emojis?.length) return html;
-  for (const emoji of emojis) {
-    const re = new RegExp(`:${emoji.shortcode}:`, 'g');
-    html = html.replace(re,
-      `:${emoji.shortcode}:`
-    );
-  }
-  return html;
-}
-```
-
-CSS: `.ap-custom-emoji { height: 1.2em; vertical-align: middle; display: inline; }`
-
----
-
-## 2. Quote Posts
-
-### How Elk Handles Quotes
-
-- Dedicated `StatusQuote.vue` component
-- Handles **7 quote states**: pending, revoked, deleted, blocked_account, blocked_domain, muted_account, rejected, accepted
-- Only renders full quote embed for `accepted` state
-- Renders as a nested `StatusCard` inside a `
` element
-- Supports shallow quotes (fetch on render) and pre-embedded quotes
-- Nesting limit: shows full card for levels 0-2, then author-only for 3+
-
-### How Phanpy Handles Quotes
-
-- `QuoteStatus` / `ShallowQuote` components
-- Full quote chain unwrapping (follows `quotedStatusId` up to 30 levels!)
-- Handles unfulfilled states (deleted, blocked, muted) with icon + message + optional "Show anyway" button
-- Marks quote links in parent content with `is-quote` CSS class (to visually distinguish them)
-- Nesting limit: level 3+ shows `@author …` only
-- State tracked in Valtio: `states.statusQuotes[statusKey]`
-
-### What We Should Adopt
-
-| Feature | Status | Priority |
-|---------|--------|----------|
-| Basic quote embed (author, content, photo) | ✅ Done (v2.4.3) | — |
-| Strip RE: link when quote renders | ✅ Done (v2.4.2) | — |
-| Quote state handling (deleted, pending) | ❌ We show stale/broken embeds | Medium |
-| Mark quote links in content CSS | ❌ Quote link looks like any other link | **High** |
-| Quote nesting depth limit | ❌ No nesting at all yet | Low |
-
-### Recommended Action: Quote Link Styling
-
-When we strip the `RE: ` paragraph, the remaining content is clean. But if we DON'T strip it (e.g., quote not yet fetched), the link should look distinct. Phanpy adds `is-quote` class. We could do this in `sanitizeContent` or in the template.
-
----
-
-## 3. Media Rendering
-
-### Elk's Media System
-
-- **Grid layouts**: 1 item = full width, 2 = 50/50, 3-4 = 2-column grid
-- **Focus point cropping**: Uses `meta.focus.x/y` for intelligent CSS `object-position`
-- **Blurhash placeholders**: Generates colored placeholder from blurhash until image loads
-- **Progressive loading**: Blurhash → low-res → full-res
-- **Lightbox**: Full-screen modal with arrow navigation, counter, alt text display
-- **Alt text badge**: "ALT" badge on images with descriptions, click to expand
-- **Aspect ratio clamping**: Between 0.8 and 6.0 to prevent extreme shapes
-- **Data saving mode**: Blur images until explicit click to load
-- **Video autoplay**: IntersectionObserver at 75% visibility, respects reduced-motion preference
-
-### Phanpy's Media System
-
-- **Grid**: `media-eq1` through `media-gt4` CSS classes
-- **QuickPinchZoom**: Mobile pinch-to-zoom on images
-- **Blurhash**: Average color extracted as background during load
-- **Focal point**: CSS custom property `--original-aspect-ratio`
-- **Media carousel**: Swipe navigation with snap scroll, RTL support
-- **ALT badges**: Indexed "ALT¹", "ALT²" for multiple media
-- **Audio/video**: Full HTML5 controls, no autoplay, preload metadata
-
-### What We Have vs. What We're Missing
-
-| Feature | Elk | Phanpy | Us | Priority |
-|---------|-----|--------|-----|----------|
-| Photo grid (1-4+) | ✅ 2-column adaptive | ✅ CSS class-based | ✅ Grid with +N badge | — |
-| Lightbox | ✅ Modal + carousel | ✅ Pinch zoom | ✅ Alpine.js overlay | — |
-| Blurhash placeholder | ✅ Canvas decode | ✅ OffscreenCanvas | ❌ No placeholder | Medium |
-| Focus point crop | ✅ object-position | ✅ CSS custom prop | ❌ Center crop only | Medium |
-| ALT text indicator | ✅ Badge + dropdown | ✅ Indexed badges | ❌ Not shown | **High** |
-| Video autoplay/pause | ✅ IntersectionObserver | ✅ Auto-pause on scroll | ❌ Manual only | Low |
-| Aspect ratio clamping | ✅ 0.8–6.0 range | ✅ Custom property | ❌ Max-height only | Low |
-
-### Recommended Action: ALT Text Badges
-
-Both clients prominently show ALT text availability. This is an accessibility feature and visual polish win.
-
-```njk
-{# In ap-item-media.njk, on each image #}
-{% if photo.alt or photo.description %}
-  ALT
-{% endif %}
-```
-
-Note: Our current data model stores photos as URL strings, not objects with alt text. We'd need to change `extractObjectData()` to store `{ url, alt, blurhash, width, height }` objects.
-
----
-
-## 4. Infinite Scroll / Pagination
-
-### Elk's Approach
-
-- **Virtual scroller** (optional): `vue-virtual-scroller` renders only visible items
-- **Stream integration**: WebSocket pushes new posts in real-time
-- **New posts banner**: Collected in `prevItems`, shown as "X new items" button
-- **Buffering**: Next page items held until buffer reaches 10, then batch-inserted
-- **End anchor**: Loads next page when within 2x viewport height of bottom
-
-### Phanpy's Approach
-
-- **IntersectionObserver** with rootMargin = 1.5x screen height
-- **Debounced loading** (1s) prevents rapid re-requests
-- **Skeleton loaders** during fetch
-- **"Show more..." button** as fallback inside observer target
-- **Auto-refresh**: Polls periodically if user is near top and window is visible
-
-### Our Current Approach
-
-- **IntersectionObserver** with rootMargin = 200px
-- **Cursor-based pagination** with `before` parameter
-- **New posts banner** polling every 30s
-- **No virtual scrolling** — all cards in DOM
-- **No skeleton loaders** — button text changes to "Loading..."
-
-### What We're Missing
-
-| Feature | Elk | Phanpy | Us | Priority |
-|---------|-----|--------|-----|----------|
-| IntersectionObserver auto-load | ✅ | ✅ 1.5x screen | ✅ 200px margin | — |
-| Manual "Load more" button | ✅ | ✅ | ✅ (just added for tabs) | — |
-| Skeleton loaders | ✅ | ✅ | ❌ Text "Loading..." only | Medium |
-| New posts banner | ✅ WebSocket stream | ✅ Polling | ✅ Polling 30s | — |
-| Virtual scrolling | ✅ Optional | ❌ | ❌ | Low (server-rendered) |
-| Debounced loading | ❌ | ✅ 1s debounce | ❌ | Low |
-
-### Recommended: Larger IntersectionObserver Margin
-
-Our 200px rootMargin means auto-load triggers late. Both clients use 1.5-2x viewport height. Easy fix:
-
-```js
-{ rootMargin: `0px 0px ${window.innerHeight}px 0px` }
-```
-
----
-
-## 5. Content Warnings / Sensitive Content
-
-### Elk's System
-
-- Separate toggles for text spoiler vs. sensitive media
-- User preferences: `expandCWByDefault`, `expandMediaByDefault`
-- Content filter integration (server-side filters shown as CW)
-- Eye icon toggle button
-- Dotted border separator between CW text and hidden content
-
-### Phanpy's System
-
-- `states.spoilers[id]` and `states.spoilersMedia[id]` — separate state per post
-- User preferences: `readingExpandSpoilers`, `readingExpandMedia`
-- Filtered content: Shows filter reason with separate reveal button
-- Three sensitivity levels: show_all, hide_all, user-controlled
-
-### Our System
-
-- Single toggle for both text and media (combined)
-- CW button with spoiler text shown
-- No user preference for auto-expand
-- Works well but lacks granularity
-
-### Gap Analysis
-
-| Feature | Elk | Phanpy | Us | Priority |
-|---------|-----|--------|-----|----------|
-| CW text toggle | ✅ | ✅ | ✅ | — |
-| Separate media toggle | ✅ | ✅ | ❌ Combined | Low |
-| Auto-expand preference | ✅ | ✅ | ❌ | Low |
-| Blurred media preview | ✅ Blurhash | ❌ | ❌ | Medium |
-
----
-
-## 6. Author Display
-
-### Elk's Approach
-
-- Display name with custom emoji
-- Handle with `@username@domain` format
-- Bot indicator icon
-- Lock (private account) indicator
-- **Hover card**: Full profile preview on mouseover (500ms delay) with bio, stats, follow button
-- Relative time ("2h ago") with absolute tooltip
-
-### Phanpy's Approach
-
-- Display name with custom emoji and bold
-- Username shown only if different from display name (smart dedup)
-- Bot accounts get squircle avatar shape
-- Role tags (moderator/admin badges)
-- **Relative time** with smart formatting
-- Punycode handling for international domains
-- RTL-safe username display with `bidi-isolate`
-
-### Our Approach
-
-- Display name (sanitized plain text)
-- Handle with `@username@domain`
-- Absolute timestamp only ("Feb 25, 2026, 4:46 PM")
-- No bot/lock indicators
-- No hover cards
-- No custom emoji in display names
-
-### What We're Missing
-
-| Feature | Elk | Phanpy | Us | Priority |
-|---------|-----|--------|-----|----------|
-| Custom emoji in names | ✅ | ✅ | ❌ Stripped to text | **High** (same fix as content emoji) |
-| Relative timestamps | ✅ "2h ago" | ✅ Smart format | ❌ Absolute only | **High** |
-| Bot/lock indicators | ✅ Icons | ✅ Squircle avatar | ❌ | Low |
-| Profile hover cards | ✅ Full card | ❌ | ❌ | Low (needs significant JS) |
-
-### Recommended Action: Relative Timestamps
-
-Both clients use relative time for in-feed cards. This is a major readability improvement. Since we server-render, we have two options:
-
-**Option A: Server-side relative time** — Compute in controller, but goes stale.
-**Option B: Client-side via Alpine** — Use a small Alpine component that converts ISO dates to relative strings. This is what both Elk and Phanpy do (client-side).
-
-```js
-// Small Alpine directive or component
-Alpine.directive('relative-time', (el) => {
-  const iso = el.getAttribute('datetime');
-  const update = () => { el.textContent = formatRelative(iso); };
-  update();
-  el._interval = setInterval(update, 60000);
-});
-```
-
----
-
-## 7. Hashtag Rendering
-
-### Elk
-
-- Hashtags in content preserved as links
-- `TagHoverWrapper` shows usage stats on hover
-- Sanitizer allows `hashtag` CSS class through
-
-### Phanpy
-
-- Spanifies: `#hashtag` inside link
-- Detects **hashtag stuffing** (3+ tags in one paragraph) → collapses with tooltip
-- Separate hashtag tags section at bottom of post (from API `tags` array, deduped against content)
-
-### Us
-
-- Hashtags extracted to `category` array, rendered as linked tags below content
-- Content HTML hashtag links pass through sanitization
-- **Bug found:** Inside `-webkit-line-clamp` containers (quote embeds), the `#tag` structure breaks because `-webkit-box` makes spans block-level (fixed in v2.4.5)
-
-### Recommended Action
-
-Our hashtag rendering is adequate. The main improvement would be Phanpy's hashtag stuffing collapse — but it's low priority since our tag rendering already extracts tags to a footer section.
-
----
-
-## 8. Interaction UI
-
-### Elk
-
-- **4 buttons**: Reply (blue), Boost (green), Quote (purple), Favorite (rose/yellow)
-- **Counts** shown per button (configurable to hide)
-- **Color-coded hover states**: Each button tints its area on hover
-- **Keyboard shortcuts**: r=reply, b=boost, f=favorite
-- **Bookmark** as 5th action
-
-### Phanpy
-
-- **4 buttons**: Reply, Boost, Like, Bookmark
-- **StatusButton component** with dual title (checked/unchecked)
-- **Shortened counts**: "123K" for large numbers
-- **Keyboard shortcuts**: r, b, f, m
-
-### Us
-
-- **5 buttons**: Reply (link), Boost (toggle), Like (toggle), View Original, Save (optional)
-- Optimistic UI with revert on error
-- CSRF-protected POSTs
-- No keyboard shortcuts
-- No counts shown
-
-### Gap Analysis
-
-| Feature | Elk | Phanpy | Us | Priority |
-|---------|-----|--------|-----|----------|
-| Like/Boost/Reply | ✅ | ✅ | ✅ | — |
-| Interaction counts | ✅ Per-button | ✅ Shortened | ❌ | Medium |
-| Keyboard shortcuts | ✅ | ✅ | ❌ | Low |
-| Color-coded buttons | ✅ | ✅ | Partial (active states) | Low |
-| Bookmark | ✅ | ✅ | ✅ (Save) | — |
-| Quote button | ✅ | ❌ | ❌ | Low |
-
----
-
-## Priority Improvements — Ranked by Impact
-
-### Tier 1: High Impact, Moderate Effort
-
-1. **Custom emoji rendering** — Both clients treat this as essential. Affects display names AND post content. Single utility function applicable everywhere.
-
-2. **Relative timestamps** — Both clients use this. Major readability improvement for timeline scanning. Small Alpine component.
-
-3. **ALT text badges on media** — Both clients show this prominently. Accessibility win. Requires enriching photo data model from URL strings to objects.
-
-4. **Quote link styling in content** — When `RE:` link isn't stripped (pending quote), distinguish it visually. CSS-only change.
-
-### Tier 2: Medium Impact, Moderate Effort
-
-5. **Skeleton loaders** for pagination — Replace "Loading..." text with card-shaped placeholder skeletons. CSS-only.
-
-6. **Blurhash placeholders** for media — Show colored placeholder while images load. Requires storing blurhash data from API.
-
-7. **Focus point cropping** — Use focal point data for smarter image crops. Requires storing focus data.
-
-8. **Interaction counts** — Show like/boost/reply counts on buttons. Data already available from API.
-
-### Tier 3: Lower Impact or High Effort
-
-9. **Hashtag stuffing collapse** — Collapse posts that are mostly hashtags.
-10. **Long URL shortening** — Truncate displayed URLs in content.
-11. **Bot/lock indicators** — Show account type badges.
-12. **Keyboard shortcuts** — Navigation and interaction hotkeys.
-13. **Video autoplay/pause on scroll** — IntersectionObserver for video elements.
-14. **Quote state handling** (deleted, pending, blocked) — Show appropriate message instead of broken embed.
-15. **Profile hover cards** — Full profile preview on author hover (significant JS investment).
-
----
-
-## Data Model Gaps
-
-Our timeline items store minimal data compared to what Elk/Phanpy consume. Key missing fields:
-
-| Field | Source | Used For |
-|-------|--------|----------|
-| `emojis[]` | `status.emojis` | Custom emoji rendering in content + names |
-| `media[].alt` | `attachment.description` | ALT text badges |
-| `media[].blurhash` | `attachment.blurhash` | Placeholder images |
-| `media[].focus` | `attachment.meta.focus` | Smart cropping |
-| `media[].width/height` | `attachment.meta.original` | Aspect ratio |
-| `repliesCount` | `status.replies_count` | Interaction counts |
-| `reblogsCount` | `status.reblogs_count` | Interaction counts |
-| `favouritesCount` | `status.favourites_count` | Interaction counts |
-| `account.bot` | `account.bot` | Bot indicator |
-| `account.emojis` | `account.emojis` | Custom emoji in display names |
-| `poll` | `status.poll` | Poll rendering |
-| `editedAt` | `status.edited_at` | Edit indicator |
-
-For **inbox-received posts** (via ActivityPub), some of these map to Fedify object properties. For **explore view posts** (via Mastodon REST API), all fields are directly available in the status object.
-
----
-
-## Architectural Constraints
-
-Our server-rendered approach means we can't do everything Elk and Phanpy do:
-
-1. **No reactive state** — We can't update a card's like count in real-time without a page refresh or AJAX call
-2. **No virtual scrolling** — All cards are in the DOM (but server-rendered HTML is lighter than React/Vue vDOM)
-3. **No hover cards** — Would require significant Alpine.js investment and API endpoints
-4. **No WebSocket streaming** — We poll instead (already have 30s new posts banner)
-
-But we have advantages too:
-- **Faster initial load** — Server-rendered HTML is immediately visible
-- **Works without JS** — Basic reading works even if Alpine fails
-- **Simpler deployment** — No build step, no client bundle
-- **Lower maintenance** — No framework version churn