Arcology Engine

Arcology Feed Publishing

Contents

Introduction

This document is the source of truth for Arcology's Atom feed publishing — a port of arcology-django's Feed / FeedEntry models (arcology-django arcology/models.py) to the KMP stack. A feed aggregates published org content into an Atom XML document served at a .xml URL, consumed by feed readers and scrapers that have no session and no base URL.

The Django original works like this: a file declares a feed with #+ARCOLOGY_FEED: site/name.xml; the headings of that file carrying a PUBDATE property become FeedEntry rows; the feed view renders the top 10 entries by pubdate into an Atom template. This port keeps the core but generalizes the model in three ways, all motivated by the same principle:

Many files can publish to one feed, the same way many headings can publish to one page URL.

Semantics

Declaration

An ARCOLOGY_FEED mention — either a file-level keyword line (#+ARCOLOGY_FEED: site/name.xml) or a heading drawer property (:ARCOLOGY_FEED: site/name.xml) — contributes a declarer row to the feeds table, keyed (route_key, file). There is no single owner: any file may declare the feed, and a feed lives as long as at least one declarer remains. Multiple files declaring the same feed each contribute a row; feed metadata is coalesced at serve time (see FeedPublisher).

The declaring node's own ARCOLOGY_KEY (drawer property, or the file's ARCOLOGY_KEY keyword line / file property when the declarer is file-level) is stored as the feed's anchor anchor_site / anchor_path — the published page this feed hangs off. The feed's title is the declaring node's title; AUTHOR comes from the declaring file's preamble; ARCOLOGY_TOOT_VISIBILITY (default private) is stored for the Phase 9 federation work.

Entry candidacy: property rollup

Entries are decided by a property rollup over the parsed document — one rule, applied uniformly, with no special cases per source:

A heading with a :PUBDATE: property becomes an entry of the feed scoped to it: the feed declared on the heading itself, else the feed declared on its nearest ancestor heading, else the file-level feed(s) declared by ARCOLOGY_FEED keyword lines. A heading declaring ARCOLOGY_FEED while carrying PUBDATE is a self-entry of the feed it declares. A section that declares its own feed carves its subtree out of the file-level feed's scope.

This one rule covers every binding case:

  • The Django shape — file declares one feed, PUBDATE headings under it are entries — falls out when only keyword lines exist.

  • Multi-feed files don't cross-contaminate: a section declaring garden/notes.xml pulls its subtree's PUBDATE headings out of the file-level feed's scope (replacement semantics, nearest ancestor-or-self).

  • Subscribing a heading in any file to a feed is just mentioning the feed on (or above) that heading.

:PUBDATE: is required for every entry, whatever its source: a mention declares a feed, a PUBDATE makes an entry. Values are parsed by PubdateParser (accepting ==, 2023-01-02 Mon 17:19, date-only, and RFC-3339 passthrough) and normalized to RFC-3339 UTC at index time; unparseable values skip the heading. Headings tagged with any tag in EXCLUDE_TAGS (noexport NOEXPORT ARCHIVE) are skipped.

The file-level node can be an entry too: a :PUBDATE: in the preamble properties drawer rolls up to the file-level feeds.

Path aggregation (serve time)

Because many files publish to one page path, a feed anchored at SITE/path also collects entries from other files: at serve time, the FeedPublisher unions the index-time feed_entries rows with nodes whose published route path matches anchor_path (the same route lookup the anchor page itself uses), keeping those nodes that carry a parseable :PUBDATE: and are not file-level (level-0) nodes. This is the journal model: a feed declared on garden/shitposts collects every heading across the org publishing to garden/shitposts, and a journal entry can join the feed by carrying :ARCOLOGY_KEY: garden/shitposts + :PUBDATE:, or subscribe directly with :ARCOLOGY_FEED: garden/shitposts.xml + :PUBDATE: — the two properties compose freely on one heading.

Serving

FeedPublisher coalesces declarer metadata (preferring the declarer whose anchor page path equals the feed path minus .xml — the ARCOLOGY_KEY: garden/shitposts / ARCOLOGY_FEED: garden/shitposts.xml pairing convention — else first by file path), unions the three entry sources, dedupes by node ID (index-time rows win), sorts by pubdate descending, and takes the top 10. Zero entries → 404, like Django's Http404. Entry HTML is rendered with the shared node-body pipeline (renderParsedNode) using a CrossDomainLinkResolver in the "unknown host" mode (=server.org: currentSite null) so every == link inside an entry resolves to an absolute URL — feed readers have no base URL, so relative links would break. Entry bodies are cached in the HtmlCache keyed on file hash plus a digest of the published-route graph, since resolved links depend on the whole route table.

Entry <link> is the node's own published route https://{domain}/{path}#{nodeId} when it has one, else the anchor page + #{nodeId}.

for now all the urls are http until the proper live deployment is done.

Deliberate divergences from Django

  • Django takes only the first ARCOLOGY_FEED keyword (next(iter(...))); every mention is honored here, matching how ARCOLOGY_KEY routes accumulate.

  • Django binds every PUBDATE heading of the declaring file to the feed (file-flat); the property rollup scopes by nearest ancestor-or-self instead, degrading to Django's behavior for single-feed files.

  • EXCLUDE_TAGS filtering happens at index time here; Django filters noexport at render time via ExportOptions.ignore_tags.

  • Declarers are stored per-file and coalesced at serve time; Django has one global Feed row. Deleting the last declarer's mention retires the feed; dangling feed_entries rows from subscriber files are invisible until those files reindex — acceptable for a sync-to-publish system.

Known parser gaps

One orgmode-kmp behavior discovered while building this (documented here so a future parser fix can close it):

  • A document-final section's property drawer is dropped when the file ends at the :END: line — the section-body parser needs a following token to flush. Real org files end with body text, so this only bites synthetic fixtures; the rollup sees no properties for such a section.

Database Schema

Two tables in Feeds.sq, mirroring =Publishing.sq='s denormalized style: populated by ArcologyFeedPlugin at index time, replaced wholesale per file on re-index, dropped when the file is removed. feeds holds one row per (feed, declarer file); feed_entries holds one row per (entry node, feed). Pubdates are stored as RFC-3339 UTC strings — lexicographic order is chronological order.

sql#+name: feeds-schema
-- Publishing: Atom feeds. One row per (feed, declaring file).
-- Populated by ArcologyFeedPlugin at index time. A feed is "alive" while at
-- least one declarer row exists; metadata is coalesced at serve time.
CREATE TABLE IF NOT EXISTS feeds (
    route_key TEXT NOT NULL,
    site TEXT NOT NULL,
    path TEXT NOT NULL,
    file TEXT NOT NULL,
    title TEXT,
    author TEXT,
    anchor_site TEXT,
    anchor_path TEXT,
    visibility TEXT NOT NULL DEFAULT 'private',
    PRIMARY KEY (route_key, file)
);

CREATE INDEX IF NOT EXISTS idx_feeds_site ON feeds (site);

-- Publishing: feed entries. One row per (entry node, feed).
-- pubdate is an RFC-3339 UTC string; lexicographic = chronological.
CREATE TABLE IF NOT EXISTS feed_entries (
    node_id TEXT NOT NULL,
    feed_route_key TEXT NOT NULL,
    title TEXT,
    pubdate TEXT NOT NULL,
    file TEXT NOT NULL,
    PRIMARY KEY (node_id, feed_route_key)
);

CREATE INDEX IF NOT EXISTS idx_feed_entries_feed ON feed_entries (feed_route_key);
sql#+name: feeds-queries
selectFeedsByRouteKey:
SELECT * FROM feeds WHERE route_key = ? ORDER BY file;

selectAllFeeds:
SELECT * FROM feeds;

selectFeedsBySite:
SELECT * FROM feeds WHERE site = ? ORDER BY route_key, file;

insertFeed:
INSERT OR REPLACE INTO feeds (route_key, site, path, file, title, author, anchor_site, anchor_path, visibility)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?);

deleteFeedsByFile:
DELETE FROM feeds WHERE file = ?;

countFeeds:
SELECT COUNT(*) FROM feeds;

selectFeedEntriesByRouteKey:
SELECT * FROM feed_entries WHERE feed_route_key = ? ORDER BY pubdate DESC;

selectAllFeedEntries:
SELECT * FROM feed_entries;

insertFeedEntry:
INSERT OR REPLACE INTO feed_entries (node_id, feed_route_key, title, pubdate, file)
VALUES (?, ?, ?, ?, ?);

deleteFeedEntriesByFile:
DELETE FROM feed_entries WHERE file = ?;

countFeedEntries:
SELECT COUNT(*) FROM feed_entries;

PubdateParser

Org timestamps in the wild carry an optional <> wrapper, an optional day-of-week name, and an optional HH:MM[:SS] time. PubdateParser normalizes all of these to RFC-3339 UTC strings, treating the naive org time as UTC — the server's own clock discipline covers the rest, and lexicographic sort stays chronological. RFC-3339 input passes through untouched.

kotlin#+name: pubdate-parser
package computer.whatthefuck.arcology.publishing

/**
 * Normalizes org timestamp strings to RFC-3339 UTC.
 *
 * Accepts the forms found in real org files:
 *  - `<2023-06-14 Wed 10:30>`  (bracketed, day-of-week, HH:MM)
 *  - `2023-01-02 Mon 17:19`    (bare, day-of-week, HH:MM)
 *  - `<2026-09-08 Tue>`        (date only → midnight)
 *  - `2026-09-08T10:15:00Z`    (RFC-3339 passthrough)
 *
 * Returns null for anything unparseable; callers skip the entry (PUBDATE is
 * strict — see the Semantics section).
 */
object PubdateParser {
    private val orgTimestamp = Regex(
        """^(\d{4})-(\d{2})-(\d{2})(?:\s+\w{3,})?(?:\s+(\d{1,2}):(\d{2})(?::(\d{2}))?)?$"""
    )
    private val rfc3339 = Regex(
        """^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:Z|[+-]\d{2}:\d{2})$"""
    )

    fun parse(raw: String): String? {
        val s = raw.trim().removePrefix("<").removeSuffix(">").trim()
        if (rfc3339.matches(s)) return s
        val m = orgTimestamp.find(s) ?: return null
        val month = m.groupValues[2].toInt()
        val day = m.groupValues[3].toInt()
        val hour = m.groupValues[4].ifEmpty { "0" }.toInt()
        val minute = m.groupValues[5].ifEmpty { "0" }.toInt()
        val second = m.groupValues[6].ifEmpty { "0" }.toInt()
        // Range-check before formatting: the regex is shape-only, and
        // formatting doesn't validate (2026-13-45 must not produce a value).
        if (month !in 1..12 || day !in 1..31 || hour > 23 || minute > 59 || second > 59) return null
        val date = "%s-%s-%s".format(m.groupValues[1], m.groupValues[2], m.groupValues[3])
        return "%sT%02d:%02d:%02dZ".format(date, hour, minute, second)
    }
}

Tests: PubdateParser

kotlin#+name: pubdate-parser-test
package computer.whatthefuck.arcology.publishing

import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertNull

class PubdateParserTest {
    @Test
    fun `parses bracketed org timestamp with day of week`() {
        assertEquals("2023-06-14T10:30:00Z", PubdateParser.parse("<2023-06-14 Wed 10:30>"))
    }

    @Test
    fun `parses bare org timestamp`() {
        assertEquals("2023-01-02T17:19:00Z", PubdateParser.parse("2023-01-02 Mon 17:19"))
    }

    @Test
    fun `parses date-only timestamp as midnight`() {
        assertEquals("2026-09-08T00:00:00Z", PubdateParser.parse("<2026-09-08 Tue>"))
    }

    @Test
    fun `passes through rfc3339`() {
        assertEquals("2026-09-08T10:15:00Z", PubdateParser.parse("2026-09-08T10:15:00Z"))
        assertEquals("2026-09-08T10:15:00-07:00", PubdateParser.parse("2026-09-08T10:15:00-07:00"))
    }

    @Test
    fun `returns null for garbage`() {
        assertNull(PubdateParser.parse("sometime next week"))
        assertNull(PubdateParser.parse(""))
        assertNull(PubdateParser.parse("2026-13-45 Mon 99:99"))
    }
}

Data Models

FeedModel is one declarer row (note: not "the feed" — the feed is the set of declarers sharing a routeKey). FeedEntryModel is one index-time entry row.

kotlin#+name: feed-models
package computer.whatthefuck.arcology.publishing

data class FeedModel(
    val routeKey: String,
    val site: String,
    val path: String,
    val file: String,
    val title: String?,
    val author: String? = null,
    val anchorSite: String? = null,
    val anchorPath: String? = null,
    val visibility: String = "private"
)

data class FeedEntryModel(
    val nodeId: String,
    val feedRouteKey: String,
    val title: String?,
    val pubdate: String,
    val file: String
)

FeedRepository

Interface and SQLDelight implementation, following the PublishingRepository pattern.

kotlin#+name: feed-repo-interface
package computer.whatthefuck.arcology.publishing

import computer.whatthefuck.arcology.db.ArcologyDatabase
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.ExperimentalCoroutinesApi
import kotlinx.coroutines.withContext

interface FeedRepository {
    suspend fun getFeedsByRouteKey(routeKey: String): List<FeedModel>
    suspend fun getAllFeeds(): List<FeedModel>
    suspend fun getFeedsBySite(site: String): List<FeedModel>
    suspend fun insertFeed(feed: FeedModel)
    suspend fun deleteFeedsByFile(file: String)
    suspend fun countFeeds(): Long

    suspend fun getEntriesForFeed(routeKey: String): List<FeedEntryModel>
    suspend fun getAllEntries(): List<FeedEntryModel>
    suspend fun insertEntry(entry: FeedEntryModel)
    suspend fun deleteEntriesByFile(file: String)
    suspend fun countEntries(): Long
}
kotlin#+name: feed-repo-impl
class FeedRepositoryImpl(
    private val database: ArcologyDatabase
) : FeedRepository {

    @OptIn(ExperimentalCoroutinesApi::class)
    private val dbDispatcher = Dispatchers.IO.limitedParallelism(1)

    private fun computer.whatthefuck.arcology.db.Feeds.toFeedModel(): FeedModel {
        return FeedModel(
            routeKey = route_key,
            site = site,
            path = path,
            file = file_,
            title = title,
            author = author,
            anchorSite = anchor_site,
            anchorPath = anchor_path,
            visibility = visibility
        )
    }

    private fun computer.whatthefuck.arcology.db.Feed_entries.toFeedEntryModel(): FeedEntryModel {
        return FeedEntryModel(
            nodeId = node_id,
            feedRouteKey = feed_route_key,
            title = title,
            pubdate = pubdate,
            file = file_
        )
    }

    override suspend fun getFeedsByRouteKey(routeKey: String): List<FeedModel> {
        return withContext(Dispatchers.IO) {
            database.feedsQueries.selectFeedsByRouteKey(routeKey).executeAsList().map { it.toFeedModel() }
        }
    }

    override suspend fun getAllFeeds(): List<FeedModel> {
        return withContext(Dispatchers.IO) {
            database.feedsQueries.selectAllFeeds().executeAsList().map { it.toFeedModel() }
        }
    }

    override suspend fun getFeedsBySite(site: String): List<FeedModel> {
        return withContext(Dispatchers.IO) {
            database.feedsQueries.selectFeedsBySite(site).executeAsList().map { it.toFeedModel() }
        }
    }

    override suspend fun insertFeed(feed: FeedModel) {
        database.feedsQueries.insertFeed(
            feed.routeKey,
            feed.site,
            feed.path,
            feed.file,
            feed.title,
            feed.author,
            feed.anchorSite,
            feed.anchorPath,
            feed.visibility
        )
    }

    override suspend fun deleteFeedsByFile(file: String) {
        database.feedsQueries.deleteFeedsByFile(file)
    }

    override suspend fun countFeeds(): Long {
        return withContext(Dispatchers.IO) {
            database.feedsQueries.countFeeds().executeAsOne()
        }
    }

    override suspend fun getEntriesForFeed(routeKey: String): List<FeedEntryModel> {
        return withContext(Dispatchers.IO) {
            // SQLDelight groups all queries from Feeds.sq — including the
            // feed_entries ones — into the single FeedsQueries property.
            database.feedsQueries.selectFeedEntriesByRouteKey(routeKey).executeAsList().map { it.toFeedEntryModel() }
        }
    }

    override suspend fun getAllEntries(): List<FeedEntryModel> {
        return withContext(Dispatchers.IO) {
            database.feedsQueries.selectAllFeedEntries().executeAsList().map { it.toFeedEntryModel() }
        }
    }

    override suspend fun insertEntry(entry: FeedEntryModel) {
        database.feedsQueries.insertFeedEntry(
            entry.nodeId,
            entry.feedRouteKey,
            entry.title,
            entry.pubdate,
            entry.file
        )
    }

    override suspend fun deleteEntriesByFile(file: String) {
        database.feedsQueries.deleteFeedEntriesByFile(file)
    }

    override suspend fun countEntries(): Long {
        return withContext(Dispatchers.IO) {
            database.feedsQueries.countFeedEntries().executeAsOne()
        }
    }
}

Tests: FeedRepository

End-to-end against the in-memory database, like the other repository tests.

kotlin#+name: feed-repo-test
package computer.whatthefuck.arcology.publishing

import computer.whatthefuck.arcology.database.DatabaseFactory
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertTrue
import kotlinx.coroutines.runBlocking

class FeedRepositoryTest {
    @Test
    fun `insert and query feeds by route key`() = runBlocking {
        val repo = FeedRepositoryImpl(DatabaseFactory.createInMemoryDatabase())
        repo.insertFeed(FeedModel("garden/shitposts.xml", "garden", "shitposts.xml", "a.org", "Shitposts", anchorSite = "garden", anchorPath = "shitposts"))
        repo.insertFeed(FeedModel("garden/shitposts.xml", "garden", "shitposts.xml", "b.org", "Also", anchorSite = null, anchorPath = null))
        val feeds = repo.getFeedsByRouteKey("garden/shitposts.xml")
        assertEquals(2, feeds.size)
        assertEquals("a.org", feeds.first().file)
    }

    @Test
    fun `delete by file removes only that declarer`() = runBlocking {
        val repo = FeedRepositoryImpl(DatabaseFactory.createInMemoryDatabase())
        repo.insertFeed(FeedModel("garden/x.xml", "garden", "x.xml", "a.org", null))
        repo.insertFeed(FeedModel("garden/x.xml", "garden", "x.xml", "b.org", null))
        repo.deleteFeedsByFile("a.org")
        assertEquals(listOf("b.org"), repo.getFeedsByRouteKey("garden/x.xml").map { it.file })
    }

    @Test
    fun `entries ordered by pubdate descending`() = runBlocking {
        val repo = FeedRepositoryImpl(DatabaseFactory.createInMemoryDatabase())
        repo.insertEntry(FeedEntryModel("n1", "f.xml", "old", "2023-01-01T00:00:00Z", "a.org"))
        repo.insertEntry(FeedEntryModel("n2", "f.xml", "new", "2024-01-01T00:00:00Z", "a.org"))
        val entries = repo.getEntriesForFeed("f.xml")
        assertEquals(listOf("n2", "n1"), entries.map { it.nodeId })
    }

    @Test
    fun `delete entries by file`() = runBlocking {
        val repo = FeedRepositoryImpl(DatabaseFactory.createInMemoryDatabase())
        repo.insertEntry(FeedEntryModel("n1", "f.xml", null, "2023-01-01T00:00:00Z", "a.org"))
        repo.deleteEntriesByFile("a.org")
        assertTrue(repo.getEntriesForFeed("f.xml").isEmpty())
        assertEquals(0, repo.countEntries())
    }
}

ArcologyFeedPlugin

The indexer plugin. Follows ArcologyPublishingPlugin's three-phase shape: delete existing rows for the file, collect sources, resolve and insert. Declarations come from both mention forms; entries come from the property rollup walk over document.content.

kotlin#+name: feed-plugin
package computer.whatthefuck.arcology.publishing

import computer.whatthefuck.arcology.indexer.IndexerPlugin
import computer.whatthefuck.arcology.parser.ParseResult
import xyz.lepisma.orgmode.OrgSection

class ArcologyFeedPlugin(
    private val repository: FeedRepository,
    private val debug: Boolean = false
) : IndexerPlugin {

    private data class FeedDeclaration(val routeKey: String, val declaringNodeId: String)

    override suspend fun onFileIndexed(result: ParseResult.Success) {
        val filePath = result.file.path
        repository.deleteFeedsByFile(filePath)
        repository.deleteEntriesByFile(filePath)

        val document = result.document ?: return
        val fileLevelNodeId = result.nodes.firstOrNull { it.level == 0 }?.id
        val nodeMap = result.nodes.associateBy { it.id }
        val tagsByNode = result.tags.groupBy { it.nodeId }.mapValues { v -> v.value.map { it.tag } }

        <<feed-plugin-declarations>>
        <<feed-plugin-insert-feeds>>
        <<feed-plugin-rollup>>
    }

    override suspend fun onFileRemoved(filePath: String) {
        repository.deleteFeedsByFile(filePath)
        repository.deleteEntriesByFile(filePath)
    }

    <<feed-plugin-helpers>>
}

Declaration collection

Keyword lines are file-scoped (declarer file-level node); drawer properties are heading-scoped (declarer that heading). Both forms are honored; duplicates collapse.

kotlin#+name: feed-plugin-declarations:noweb-ref feed-plugin-declarations
val declarations = mutableListOf<FeedDeclaration>()
document.keywordLines.forEach { kw ->
    if (kw.keyword == "ARCOLOGY_FEED" && kw.value.isNotEmpty() && fileLevelNodeId != null) {
        declarations.add(FeedDeclaration(kw.value.trim(), fileLevelNodeId))
    }
}
result.nodeProperties.forEach { prop ->
    if (prop.key == "ARCOLOGY_FEED" && !prop.value.isNullOrEmpty()) {
        declarations.add(FeedDeclaration(prop.value.trim(), prop.nodeId))
    }
}
val seenDeclarations = mutableSetOf<Pair<String, String>>()
val dedupedDeclarations = declarations.filter { seenDeclarations.add(it.routeKey to it.declaringNodeId) }

Feed metadata insertion

Each declaration becomes a feeds row. The anchor is the declaring node's own ARCOLOGY_KEY (drawer, file property, or keyword line); visibility from ARCOLOGY_TOOT_VISIBILITY (Django's default private); author from the preamble.

kotlin#+name: feed-plugin-insert-feeds:noweb-ref feed-plugin-insert-feeds
val visibility = document.keywordLines.firstOrNull { it.keyword == "ARCOLOGY_TOOT_VISIBILITY" && it.value.isNotEmpty() }?.value?.trim()
    ?: result.nodeProperties.firstOrNull { it.key == "ARCOLOGY_TOOT_VISIBILITY" && !it.value.isNullOrEmpty() }?.value?.trim()
    ?: result.fileProperties.firstOrNull { it.key == "ARCOLOGY_TOOT_VISIBILITY" && !it.value.isNullOrEmpty() }?.value?.trim()
    ?: "private"
        val fileKeyFromKeywords = document.keywordLines.firstOrNull { it.keyword == "ARCOLOGY_KEY" && it.value.isNotEmpty() }?.value?.trim()
        val fileKeyFromProperties = result.fileProperties.firstOrNull { it.key == "ARCOLOGY_KEY" && !it.value.isNullOrEmpty() }?.value?.trim()
        // AUTHOR can land in two places depending on blank-line placement in
        // the preamble: contiguous keyword lines parse into preamble.author;
        // after a blank line the line becomes a preface OrgKeywordLine chunk
        // (hoisted onto keywordLines like ARCOLOGY_KEY is). Accept both.
        val author = document.preamble.author
            ?: document.keywordLines.firstOrNull { it.keyword == "AUTHOR" && it.value.isNotEmpty() }?.value?.trim()

for (declaration in dedupedDeclarations) {
    val key = PublishKey.parse(declaration.routeKey)
    if (key == null) {
        if (debug) println("[ArcologyFeedPlugin] $filePath: SKIP invalid feed key '${declaration.routeKey}'")
        continue
    }
    // Anchor: the declaring heading's own ARCOLOGY_KEY, else the file's.
    val anchor = result.nodeProperties
        .firstOrNull { it.nodeId == declaration.declaringNodeId && it.key == "ARCOLOGY_KEY" && !it.value.isNullOrEmpty() }
        ?.value?.trim()?.let { PublishKey.parse(it) }
        ?: fileKeyFromProperties?.let { PublishKey.parse(it) }
        ?: fileKeyFromKeywords?.let { PublishKey.parse(it) }

    if (debug) {
        println("[ArcologyFeedPlugin] $filePath: FEED ${key.site}/${key.path} declarer=${declaration.declaringNodeId} anchor=${anchor?.site}/${anchor?.path}")
    }

    repository.insertFeed(FeedModel(
        routeKey = "${key.site}/${key.path}",
        site = key.site,
        path = key.path,
        file = filePath,
        title = nodeMap[declaration.declaringNodeId]?.title,
        author = author,
        anchorSite = anchor?.site,
        anchorPath = anchor?.path,
        visibility = visibility
    ))
}

Property rollup: entry binding

Recursive walk of the document tree carrying the active feed scope. active self-declared feeds, else the inherited (nearest ancestor) scope, else the file-level feeds. PUBDATE candidates come from the node properties (plus the preamble drawer for the file-level node); a candidate joins every feed in its active scope, subject to EXCLUDE_TAGS and pubdate parsing.

kotlin#+name: feed-plugin-rollup:noweb-ref feed-plugin-rollup
val fileLevelFeedKeys = dedupedDeclarations
    .filter { it.declaringNodeId == fileLevelNodeId }
    .map { it.routeKey }
    .distinct()

// PUBDATE candidates by node id; the preamble drawer rolls up to the file-level node.
val pubdateByNode = result.nodeProperties
    .filter { it.key == "PUBDATE" && !it.value.isNullOrEmpty() }
    .associate { it.nodeId to it.value!! }
    .toMutableMap()
if (fileLevelNodeId != null) {
    result.fileProperties.firstOrNull { it.key == "PUBDATE" && !it.value.isNullOrEmpty() }?.let { fp ->
        pubdateByNode.putIfAbsent(fileLevelNodeId, fp.value!!)
    }
}

val entriesToInsert = mutableListOf<Triple<String, String, String>>() // (nodeId, feedRouteKey, pubdate)
val seenEntries = mutableSetOf<Pair<String, String>>()

fun walk(sections: List<OrgSection>, inherited: List<String>) {
    for (section in sections) {
        val nodeId = sectionNodeId(section)
        val selfDeclared = nodeId?.let { id ->
            dedupedDeclarations.filter { it.declaringNodeId == id }.map { it.routeKey }
        } ?: emptyList()
        // Replacement semantics: a section's own declaration carves its
        // subtree out of the inherited scope.
        val active = selfDeclared.ifEmpty { inherited }
        if (nodeId != null && active.isNotEmpty() && pubdateByNode.containsKey(nodeId)) {
            val tags = tagsByNode[nodeId] ?: emptyList()
            val pubdate = PubdateParser.parse(pubdateByNode[nodeId]!!)
            if (tags.none { it in EXCLUDE_TAGS } && pubdate != null) {
                for (feedKey in active) {
                    if (seenEntries.add(nodeId to feedKey)) {
                        entriesToInsert.add(Triple(nodeId, feedKey, pubdate))
                    }
                }
            }
        }
        walk(section.body.filterIsInstance<OrgSection>(), active)
    }
}
walk(document.content, fileLevelFeedKeys)

for ((nodeId, feedKey, pubdate) in entriesToInsert) {
    if (debug) println("[ArcologyFeedPlugin] $filePath: ENTRY $nodeId → $feedKey @ $pubdate")
    repository.insertEntry(FeedEntryModel(
        nodeId = nodeId,
        feedRouteKey = feedKey,
        title = nodeMap[nodeId]?.title,
        pubdate = pubdate,
        file = filePath
    ))
}

Helpers

Section ID extraction mirrors findSectionByNodeId in server.org — heading property drawers hold the node ID.

kotlin#+name: feed-plugin-helpers:noweb-ref feed-plugin-helpers
private fun sectionNodeId(section: OrgSection): String? {
    return section.heading.properties?.map?.get("ID")?.let { orgLine ->
        orgLine.items.filterIsInstance<xyz.lepisma.orgmode.OrgInlineElem.Text>()
            .joinToString("") { it.text }.trim()
    }
}

Tests: ArcologyFeedPlugin

Fixtures are parsed with the real OrgFileParser so the rollup walk runs against genuine OrgDocument trees; repositories are recording fakes.

kotlin#+name: feed-plugin-test-prelude
package computer.whatthefuck.arcology.publishing

import computer.whatthefuck.arcology.parser.OrgFileParser
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertTrue
import kotlinx.coroutines.runBlocking
import kotlin.time.Instant

class ArcologyFeedPluginTest {

    private val parser = OrgFileParser()

    private fun parse(content: String): computer.whatthefuck.arcology.parser.ParseResult.Success {
        val result = parser.parseFileContent("test.org", content, Instant.fromEpochSeconds(0))
        return result as computer.whatthefuck.arcology.parser.ParseResult.Success
    }

    private class RecordingFeedRepository : FeedRepository {
        val feeds = mutableListOf<FeedModel>()
        val entries = mutableListOf<FeedEntryModel>()
        val deletedFeedFiles = mutableListOf<String>()
        val deletedEntryFiles = mutableListOf<String>()
        override suspend fun getFeedsByRouteKey(routeKey: String) = feeds.filter { it.routeKey == routeKey }
        override suspend fun getAllFeeds() = feeds
        override suspend fun getFeedsBySite(site: String) = feeds.filter { it.site == site }
        override suspend fun insertFeed(feed: FeedModel) { feeds.add(feed) }
        override suspend fun deleteFeedsByFile(file: String) { deletedFeedFiles.add(file) }
        override suspend fun countFeeds() = feeds.size.toLong()
        override suspend fun getEntriesForFeed(routeKey: String) = entries.filter { it.feedRouteKey == routeKey }
        override suspend fun getAllEntries() = entries
        override suspend fun insertEntry(entry: FeedEntryModel) { entries.add(entry) }
        override suspend fun deleteEntriesByFile(file: String) { deletedEntryFiles.add(file) }
        override suspend fun countEntries() = entries.size.toLong()
    }
kotlin#+name: feed-plugin-test:noweb-ref feed-plugin-test
    private val djangoShaped = """
:PROPERTIES:
:ID:       20220727T154730
:END:
#+title: Shitposts
#+ARCOLOGY_KEY: garden/shitposts
#+ARCOLOGY_FEED: garden/shitposts.xml

,* first post
:PROPERTIES:
:ID:       20230614T222450
:PUBDATE:  <2023-06-14 Wed 10:30>
:END:

,* second post
:PROPERTIES:
:ID:       20230613T100501
:PUBDATE:  2023-06-13 Mon 10:00
:END:

,* no pubdate here
:PROPERTIES:
:ID:       20230612T100501
:END:
""".trimIndent()

    @Test
    fun `django shape - file feed with PUBDATE headings as entries`() = runBlocking {
        val repo = RecordingFeedRepository()
        ArcologyFeedPlugin(repo).onFileIndexed(parse(djangoShaped))
        assertEquals(1, repo.feeds.size)
        val feed = repo.feeds.first()
        assertEquals("garden/shitposts.xml", feed.routeKey)
        assertEquals("garden", feed.anchorSite)
        assertEquals("shitposts", feed.anchorPath)
        assertEquals("Shitposts", feed.title)
        // Two entries; the PUBDATE-less heading is skipped.
        assertEquals(listOf("20230614T222450", "20230613T100501"), repo.entries.map { it.nodeId })
        assertEquals("2023-06-14T10:30:00Z", repo.entries.first().pubdate)
    }
kotlin#+name: feed-plugin-test:noweb-ref feed-plugin-test
    @Test
    fun `re-index deletes existing rows for the file first`() = runBlocking {
        val repo = RecordingFeedRepository()
        ArcologyFeedPlugin(repo).onFileIndexed(parse(djangoShaped))
        assertEquals(listOf("test.org"), repo.deletedFeedFiles)
        assertEquals(listOf("test.org"), repo.deletedEntryFiles)
    }

    @Test
    fun `heading declaring feed with PUBDATE is a self-entry and carves out subtree`() = runBlocking {
        // "another note" is a *second top-level* heading: a sibling of the
        // notes section, not a child of it (nesting is by heading level, so
        // ** under * garden notes would be IN the section). It therefore
        // inherits the file-level feed, while the notes section's subtree
        // (its own PUBDATE + the nested note) rolls up to notes.xml.
        val content = """
:PROPERTIES:
:ID:       20260908T000001
:END:
#+ARCOLOGY_KEY: garden/notes
#+ARCOLOGY_FEED: garden/updates.xml

,* garden notes
:PROPERTIES:
:ID:       20260908T000002
:ARCOLOGY_FEED: garden/notes.xml
:PUBDATE:  <2026-09-08 Tue 10:00>
:END:

,** a note in the section
:PROPERTIES:
:ID:       20260908T000003
:PUBDATE:  <2026-09-07 Mon 09:00>
:END:

,* another note outside the section
:PROPERTIES:
:ID:       20260908T000004
:PUBDATE:  <2026-09-06 Sun 08:00>
:END:

Trailing body text so the final section's drawer flushes.
""".trimIndent()
        val repo = RecordingFeedRepository()
        ArcologyFeedPlugin(repo).onFileIndexed(parse(content))

        val updatesEntries = repo.entries.filter { it.feedRouteKey == "garden/updates.xml" }.map { it.nodeId }.toSet()
        val notesEntries = repo.entries.filter { it.feedRouteKey == "garden/notes.xml" }.map { it.nodeId }.toSet()
        // The notes section declares notes.xml: its own PUBDATE (self-entry)
        // and its subtree roll up to notes.xml, carved out of the file-level feed.
        assertEquals(setOf("20260908T000002", "20260908T000003"), notesEntries)
        // The sibling top-level heading inherits the file-level feed.
        assertEquals(setOf("20260908T000004"), updatesEntries)
    }

    @Test
    fun `EXCLUDE_TAGS headings do not become entries`() = runBlocking {
        val content = """
:PROPERTIES:
:ID:       20260908T000010
:END:
#+ARCOLOGY_FEED: garden/x.xml

,* archived post
:PROPERTIES:
:ID:       20260908T000011
:PUBDATE:  <2026-09-08 Tue 10:00>
:ARCHIVE:
:END:

trailing body
""".trimIndent()
        val repo = RecordingFeedRepository()
        ArcologyFeedPlugin(repo).onFileIndexed(parse(content))
        assertTrue(repo.entries.none { it.nodeId == "20260908T000011" })
    }

    @Test
    fun `unparseable PUBDATE skips the heading`() = runBlocking {
        val content = """
:PROPERTIES:
:ID:       20260908T000020
:END:
#+ARCOLOGY_FEED: garden/x.xml

,* someday maybe
:PROPERTIES:
:ID:       20260908T000021
:PUBDATE:  sometime next week
:END:

trailing body
""".trimIndent()
        val repo = RecordingFeedRepository()
        ArcologyFeedPlugin(repo).onFileIndexed(parse(content))
        assertTrue(repo.entries.isEmpty())
    }

    @Test
    fun `drawer-declared feed gets heading title and heading anchor`() = runBlocking {
        // NOTE: a trailing body line after the last :END: — the parser drops a
        // document-final section's body chunks when the file ends at the drawer.
        val content = """
:PROPERTIES:
:ID:       20260908T000030
:END:
#+ARCOLOGY_KEY: garden/index

,* my feed section
:PROPERTIES:
:ID:       20260908T000031
:ARCOLOGY_KEY: garden/feedpage
:ARCOLOGY_FEED: garden/feedpage.xml
:END:

Some body text so the section's drawer flushes.
""".trimIndent()
        val repo = RecordingFeedRepository()
        ArcologyFeedPlugin(repo).onFileIndexed(parse(content))
        val feed = repo.feeds.single()
        assertEquals("my feed section", feed.title)
        assertEquals("garden", feed.anchorSite)
        assertEquals("feedpage", feed.anchorPath)
        assertEquals("private", feed.visibility)
    }

    @Test
    fun `TOOT_VISIBILITY and AUTHOR are captured`() = runBlocking {
        // AUTHOR sits after a blank line like the real files, so it lands in
        // the preface as an OrgKeywordLine rather than the preamble's
        // author field — the plugin accepts both surfaces.
        val content = """
:PROPERTIES:
:ID:       20260908T000040
:END:
#+ARCOLOGY_KEY: garden/x

#+AUTHOR: ryan rix <garden@whatthefuck.computer>
#+ARCOLOGY_FEED: garden/x.xml
#+ARCOLOGY_TOOT_VISIBILITY: unlisted

trailing body
""".trimIndent()
        val repo = RecordingFeedRepository()
        ArcologyFeedPlugin(repo).onFileIndexed(parse(content))
        assertEquals("unlisted", repo.feeds.single().visibility)
        assertEquals("ryan rix <garden@whatthefuck.computer>", repo.feeds.single().author)
    }

    @Test
    fun `onFileRemoved deletes rows for the file`() = runBlocking {
        val repo = RecordingFeedRepository()
        ArcologyFeedPlugin(repo).onFileRemoved("gone.org")
        assertEquals(listOf("gone.org"), repo.deletedFeedFiles)
        assertEquals(listOf("gone.org"), repo.deletedEntryFiles)
    }
kotlin#+name: feed-plugin-test-end
}

FeedPublisher

The serve-time aggregator. Coalesces declarer metadata, unions index-time entries with path-aggregated nodes, dedupes, sorts, limits to 10, renders entry HTML through the shared node-body pipeline with absolute cross-domain links, and produces the feed.peb model.

kotlin#+name: feed-publisher
package computer.whatthefuck.arcology.publishing

import computer.whatthefuck.arcology.database.RoamRepository
import computer.whatthefuck.arcology.parser.OrgFileParser
import computer.whatthefuck.arcology.parser.ParseResult
import java.io.File
import java.security.MessageDigest
import kotlinx.coroutines.runBlocking

data class FeedEntryXml(
    val nodeId: String,
    val title: String,
    val url: String,
    val published: String,
    val updated: String,
    val html: String
)

data class FeedXmlModel(
    val title: String,
    val feedUrl: String,
    val pageUrl: String,
    val author: String,
    val updatedAt: String,
    val entries: List<FeedEntryXml>,
    val etag: String? = null
)

/**
 * Serves Atom feeds for ARCOLOGY_FEED-declared routes. See [[feeds.org]] for
 * the semantic model: declarer coalescing, property-rollup entries,
 * path-aggregated entries, strict PUBDATE, EXCLUDE_TAGS filtering.
 */
class FeedPublisher(
    private val feedRepository: FeedRepository,
    private val publishingRepository: PublishingRepository,
    private val repository: RoamRepository,
    private val domainMap: DomainMap,
    private val htmlCache: HtmlCache,
    private val orgDir: String,
    private val parser: OrgFileParser,
    private val debug: Boolean = false
) {
    <<feed-publisher-serve>>
    <<feed-publisher-helpers>>
}

serveFeed

Non-suspend with runBlocking inside, matching the server's servePath pattern. localhost mode consults all routes for aggregation; production consults published routes only.

kotlin#+name: feed-publisher-serve:noweb-ref feed-publisher-serve
fun serveFeed(routeKey: String, localhost: Boolean): FeedXmlModel? = runBlocking {
    val declarers = feedRepository.getFeedsByRouteKey(routeKey)
    if (declarers.isEmpty()) return@runBlocking null
    val feedKey = PublishKey.parse(routeKey) ?: return@runBlocking null
    val pathNoXml = feedKey.path.removeSuffix(".xml")

    // Coalesce metadata: prefer the declarer whose anchor page IS the feed's
    // parent page (the KEY/FEED pairing convention), else first by file path.
    val canonical = declarers
        .sortedBy { it.file }
        .firstOrNull { it.anchorPath == pathNoXml }
        ?: declarers.minBy { it.file }

    // Union entry sources, deduped by node id; index-time rows win.
    val byNode = LinkedHashMap<String, FeedEntryModel>()
    feedRepository.getEntriesForFeed(routeKey).forEach { entry ->
        byNode.putIfAbsent(entry.nodeId, entry)
    }

    val anchorPath = canonical.anchorPath
    if (anchorPath != null) {
        val routes = if (localhost) {
            publishingRepository.getRoutesByPath(anchorPath)
        } else {
            publishingRepository.getPublishedRoutesByPath(anchorPath)
        }.filter { canonical.anchorSite == null || it.site == canonical.anchorSite }
        val candidateIds = routes.map { it.nodeId }.filter { !byNode.containsKey(it) }
        val tagsByNode = repository.getTagsByNodes(candidateIds)
        for (route in routes) {
            if (byNode.containsKey(route.nodeId)) continue
            val node = repository.getNodeById(route.nodeId) ?: continue
            // Path-aggregation skips file-level (level-0) nodes: the anchor
            // page's own root should not show up as the feed's oldest entry.
            if (node.level == 0) continue
            if ((tagsByNode[route.nodeId] ?: emptyList()).any { it in EXCLUDE_TAGS }) continue
            // Strict PUBDATE for every entry source.
            val raw = repository.getHeadingProperty(route.nodeId, "PUBDATE") ?: continue
            val pubdate = PubdateParser.parse(raw) ?: continue
            byNode[route.nodeId] = FeedEntryModel(
                nodeId = route.nodeId,
                feedRouteKey = routeKey,
                title = route.title,
                pubdate = pubdate,
                file = route.file
            )
        }
    }

    val entries = byNode.values.sortedByDescending { it.pubdate }.take(10)
    if (entries.isEmpty()) return@runBlocking null

    <<feed-publisher-render>>
    <<feed-publisher-model>>
}

Entry HTML rendering

One parse per source file, N renders against a single renderer in unknown-host mode; bodies cached per node keyed on file hash + route-graph digest.

kotlin#+name: feed-publisher-render:noweb-ref feed-publisher-render
val publishedIndex = buildRouteIndex(publishingRepository.getPublishedRoutes())
val graphDigest = routeGraphDigest(publishedIndex)
val resolver = CrossDomainLinkResolver(
    routeIndex = publishedIndex,
    domainMap = domainMap,
    repository = repository,
    currentSite = null,   // unknown host: absolute links for feed readers
    localhost = false,
    publishingRepository = publishingRepository
)
val renderer = OrgHtmlRenderer(resolver, publishedIndex.keys, emptyMap(), emptyMap())
val parsedFiles = mutableMapOf<String, ParseResult.Success?>()

val xmlEntries = entries.map { entry ->
    val cacheKey = "${fileHashFor(entry.file)}:$graphDigest"
    val cached = htmlCache.getPage("feed:${entry.nodeId}", cacheKey)
            val html = cached?.html ?: run {
                val parsed = parsedFiles.getOrPut(entry.file) { loadNodeParseResult(orgDir, parser, repository, entry.nodeId, fileOverride = entry.file) }
                val body = parsed?.let { renderParsedNode(it, entry.nodeId, renderer) } ?: ""
        renderer.consumeFootnotes()
        htmlCache.putPage("feed:${entry.nodeId}", cacheKey, CachedPage(html = body))
        body
    }
    FeedEntryXml(
        nodeId = entry.nodeId,
        title = entry.title ?: entry.nodeId,
        url = entryUrl(entry.nodeId, canonical, publishedIndex),
        published = entry.pubdate,
        updated = entry.pubdate,
        html = html
    )
}

Model assembly and helpers

kotlin#+name: feed-publisher-model:noweb-ref feed-publisher-model
val feedUrl = "http://${domainMap.resolve(canonical.site) ?: "localhost"}/${canonical.path}"
val pageUrl = if (canonical.anchorSite != null && canonical.anchorPath != null) {
    "http://${domainMap.resolve(canonical.anchorSite) ?: "localhost"}/${canonical.anchorPath}"
} else {
    "http://${domainMap.resolve(canonical.site) ?: "localhost"}/${pathNoXml}"
}
val author = canonical.author
    ?: repository.getFileProperties(canonical.file)["AUTHOR"]
    ?: "Arcology User"
val etag = "W/\"${sha256Hex(entries.joinToString(",") { "${it.nodeId}:${it.pubdate}" })}\""
if (debug) println("[FeedPublisher] $routeKey: ${entries.size} entries, canonical=${canonical.file}")

FeedXmlModel(
    title = canonical.title ?: pathNoXml,
    feedUrl = feedUrl,
    pageUrl = pageUrl,
    author = author,
    updatedAt = entries.first().pubdate,
    entries = xmlEntries,
    etag = etag
)
kotlin#+name: feed-publisher-helpers:noweb-ref feed-publisher-helpers
private fun buildRouteIndex(routes: List<RouteEntry>): Map<String, RouteEntry> {
    val index = mutableMapOf<String, RouteEntry>()
    for (entry in routes) {
        if (!index.containsKey(entry.nodeId)) {
            index[entry.nodeId] = entry
        }
    }
    return index
}

// Entry URLs need the published route index built in serveFeed, so the index
// is passed in rather than stashed in publisher state.
private fun entryUrl(nodeId: String, feed: FeedModel, publishedIndex: Map<String, RouteEntry>): String {
    publishedIndex[nodeId]?.let { route ->
        domainMap.resolve(route.site)?.let { domain ->
            return "http://$domain/${route.path}#$nodeId"
        }
    }
    if (feed.anchorSite != null && feed.anchorPath != null) {
        domainMap.resolve(feed.anchorSite)?.let { domain ->
            return "http://$domain/${feed.anchorPath}#$nodeId"
        }
    }
    return "/${feed.path.removeSuffix(".xml")}#$nodeId"
}

private fun routeGraphDigest(index: Map<String, RouteEntry>): String =
    sha256Hex(index.values.map { "${it.site}/${it.path}#${it.nodeId}" }.sorted().joinToString("\n"))

private fun fileHashFor(file: String): String {
    val orgFile = File(orgDir, file)
    return if (orgFile.exists()) {
        sha256Bytes(orgFile.readBytes())
    } else {
        file.hashCode().toString()
    }
}

private fun sha256Hex(text: String): String =
    MessageDigest.getInstance("SHA-256").digest(text.toByteArray()).joinToString("") { "%02x".format(it) }

private fun sha256Bytes(bytes: ByteArray): String =
    MessageDigest.getInstance("SHA-256").digest(bytes).joinToString("") { "%02x".format(it) }

with the render block calling entryUrl(entry.nodeId, canonical, publishedIndex).

Tests: FeedPublisher

An integration test against the real in-memory database and a temp org file: index the file with both plugins, then serve the feed.

kotlin#+name: feed-publisher-test
package computer.whatthefuck.arcology.publishing

import computer.whatthefuck.arcology.database.DatabaseFactory
import computer.whatthefuck.arcology.database.RoamRepositoryImpl
import computer.whatthefuck.arcology.domain.OrgFile
import computer.whatthefuck.arcology.parser.OrgFileParser
import java.io.File
import java.nio.file.Files
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertNotNull
import kotlin.test.assertTrue
import kotlinx.coroutines.runBlocking
import kotlin.time.Instant

class FeedPublisherTest {
    @Test
    fun `serves feed with in-file entries and absolute links`() = runBlocking {
        val orgDir = Files.createTempDirectory("arcology-feed-test").toFile()
        val content = """
:PROPERTIES:
:ID:       20220727T154730
:END:
#+title: Shitposts
#+AUTHOR: ryan rix
#+ARCOLOGY_KEY: garden/shitposts
#+ARCOLOGY_FEED: garden/shitposts.xml

,* first post
:PROPERTIES:
:ID:       20230614T222450
:PUBDATE:  <2023-06-14 Wed 10:30>
:END:

body text about homestuck

,* second post
:PROPERTIES:
:ID:       20230613T100501
:PUBDATE:  <2023-06-13 Mon 10:00>
:END:
""".trimIndent()
        File(orgDir, "shitposts.org").writeText(content)

        val database = DatabaseFactory.createInMemoryDatabase()
        val roamRepository = RoamRepositoryImpl(database)
        val publishingRepository = PublishingRepositoryImpl(database)
        val feedRepository = FeedRepositoryImpl(database)

        val parser = OrgFileParser()
        val parsed = parser.parseFileContent("shitposts.org", content, Instant.fromEpochSeconds(0)) as computer.whatthefuck.arcology.parser.ParseResult.Success
        roamRepository.insertFile(OrgFile(path = "shitposts.org", hash = "abc", accessTime = Instant.fromEpochSeconds(0), modificationTime = Instant.fromEpochSeconds(0)))
        roamRepository.insertNode(computer.whatthefuck.arcology.domain.OrgNode(id = "20220727T154730", file = "shitposts.org", level = 0, position = 0, title = "Shitposts"))
        roamRepository.insertNode(computer.whatthefuck.arcology.domain.OrgNode(id = "20230614T222450", file = "shitposts.org", level = 1, position = 1, title = "first post"))
        roamRepository.insertNode(computer.whatthefuck.arcology.domain.OrgNode(id = "20230613T100501", file = "shitposts.org", level = 1, position = 2, title = "second post"))
        ArcologyPublishingPlugin(publishingRepository).onFileIndexed(parsed)
        ArcologyFeedPlugin(feedRepository).onFileIndexed(parsed)

        val publisher = FeedPublisher(
            feedRepository = feedRepository,
            publishingRepository = publishingRepository,
            repository = roamRepository,
            domainMap = DomainMap.parse("""{"sites":[{"key":"garden","domains":["garden.example.org"]}]}"""),
            htmlCache = HtmlCache(Files.createTempDirectory("arcology-feed-cache").toFile()),
            orgDir = orgDir.absolutePath,
            parser = parser
        )

        val model = publisher.serveFeed("garden/shitposts.xml", localhost = false)
        assertNotNull(model)
        assertEquals("Shitposts", model.title)
        assertEquals("ryan rix", model.author)
        assertEquals("2023-06-14T10:30:00Z", model.updatedAt)
        assertEquals(2, model.entries.size)
        assertEquals("first post", model.entries[0].title)
        // Absolute cross-domain link for feed readers.
        assertEquals("http://garden.example.org/shitposts#20230614T222450", model.entries[0].url)
        assertTrue(model.entries[0].html.contains("homestuck"))
        assertNotNull(model.etag)
    }

    @Test
    fun `returns null for unknown or empty feed`() = runBlocking {
        val publisher = FeedPublisher(
            feedRepository = FeedRepositoryImpl(DatabaseFactory.createInMemoryDatabase()),
            publishingRepository = PublishingRepositoryImpl(DatabaseFactory.createInMemoryDatabase()),
            repository = RoamRepositoryImpl(DatabaseFactory.createInMemoryDatabase()),
            domainMap = DomainMap.fromMap(emptyMap()),
            htmlCache = HtmlCache(Files.createTempDirectory("arcology-feed-cache").toFile()),
            orgDir = "/tmp",
            parser = OrgFileParser()
        )
        assertTrue(publisher.serveFeed("no/such.xml", localhost = false) == null)
    }

    @Test
    fun `feed peb renders content not summary, escaped per RFC 4287`() = runBlocking {
        // Same engine config as the server: auto-escaping on (HTML strategy is
        // Pebble's default; the escaper machinery is always registered, the
        // flag just starts false).
        val engine = io.pebbletemplates.pebble.PebbleEngine.Builder()
            .loader(io.pebbletemplates.pebble.loader.ClasspathLoader().apply { prefix = "templates/" })
            .autoEscaping(true)
            .build()
        val publisher = FeedPublisher(
            feedRepository = FeedRepositoryImpl(DatabaseFactory.createInMemoryDatabase()),
            publishingRepository = PublishingRepositoryImpl(DatabaseFactory.createInMemoryDatabase()),
            repository = RoamRepositoryImpl(DatabaseFactory.createInMemoryDatabase()),
            domainMap = DomainMap.fromMap(emptyMap()),
            htmlCache = HtmlCache(Files.createTempDirectory("arcology-feed-cache").toFile()),
            orgDir = "/tmp",
            parser = OrgFileParser()
        )
        val model = FeedXmlModel(
            title = "t",
            feedUrl = "http://s.example/f.xml",
            pageUrl = "http://s.example/p",
            author = "a",
            updatedAt = "2026-09-08T00:00:00Z",
            entries = listOf(FeedEntryXml("n1", "<b>title</b>", "http://s.example/p#n1", "2026-09-08T00:00:00Z", "2026-09-08T00:00:00Z", "<p>hello & <em>world</em></p>"))
        )
        val scope = mapOf<String, Any>(
            "title" to model.title,
            "pageUrl" to model.pageUrl,
            "feedUrl" to model.feedUrl,
            "author" to model.author,
            "updatedAt" to model.updatedAt,
            "feedEntries" to model.entries.map { entry ->
                mapOf(
                    "nodeId" to entry.nodeId,
                    "title" to entry.title,
                    "url" to entry.url,
                    "published" to entry.published,
                    "updated" to entry.updated,
                    "html" to entry.html
                )
            }
        )
        val out = java.io.StringWriter()
        engine.getTemplate("feed.peb").evaluate(out, scope)
        val xml = out.toString()
        // RFC 4287 §4.1.3: type="html" content is entity-escaped (& < > at minimum).
        assertTrue(xml.contains("<content type=\"html\">&lt;p&gt;hello &amp; &lt;em&gt;world&lt;/em&gt;&lt;/p&gt;</content>"))
        // RFC 4287 §4.2.13: summary is text-only; content lives in <content>.
        assertTrue(!xml.contains("<summary"))
        assertTrue(!xml.contains("<p>hello"))
    }

    @Test
    fun `opml lists feeds scoped to site, localhost lists all`() = runBlocking {
        val feedRepo = FeedRepositoryImpl(DatabaseFactory.createInMemoryDatabase())
        feedRepo.insertFeed(FeedModel("garden/shitposts.xml", "garden", "shitposts.xml", "a.org", "Shitposts", anchorSite = "garden", anchorPath = "shitposts"))
        feedRepo.insertFeed(FeedModel("garden/updates.xml", "garden", "updates.xml", "b.org", null))
        feedRepo.insertFeed(FeedModel("lionsrear/2023.xml", "lionsrear", "2023.xml", "c.org", "Hawaii"))
        val domainMap = DomainMap.parse("""{"sites":[{"key":"garden","title":"The Garden","domains":["garden.example.org"]},{"key":"lionsrear","title":"The Lions Rear","domains":["lionsrear.example.org"]}]}""")

        // Site-scoped: only garden's feeds, absolute URLs via the domain map.
        val siteOpml = opmlFor("garden", localhost = false, feedRepository = feedRepo, domainMap = domainMap)
        assertNotNull(siteOpml)
        assertTrue(siteOpml.contains("<title>The Garden feeds</title>"))
        assertTrue(siteOpml.contains("xmlUrl=\"http://garden.example.org/shitposts.xml\""))
        assertTrue(siteOpml.contains("htmlUrl=\"http://garden.example.org/shitposts\""))
        assertTrue(siteOpml.contains("text=\"Shitposts\""))
        assertTrue(!siteOpml.contains("lionsrear"))

        // Localhost: every site's feeds with in-server relative URLs.
        val localOpml = opmlFor(null, localhost = true, feedRepository = feedRepo, domainMap = domainMap)
        assertNotNull(localOpml)
        assertTrue(localOpml.contains("xmlUrl=\"/lionsrear/2023.xml\""))
        assertTrue(localOpml.contains("xmlUrl=\"/garden/shitposts.xml\""))

        // Unknown site: null → the route 404s.
        assertTrue(opmlFor("nosuch", localhost = false, feedRepository = feedRepo, domainMap = domainMap) == null)
        // No feed repository wired: null.
        assertTrue(opmlFor("garden", localhost = false, feedRepository = null, domainMap = domainMap) == null)
    }
}

Tangle Targets

Feeds.sq

sql#+name: feeds-sq-assembly:tangle ../src/commonMain/sqldelight/computer/whatthefuck/arcology/db/Feeds.sq:noweb yes
-- [[file:web/feeds.org::feeds-schema][feeds-schema]]

<<feeds-schema>>

<<feeds-queries>>
-- feeds-sq-assembly ends here

FeedModels.kt

kotlin#+name: feed-models-assembly:tangle ../src/commonMain/kotlin/computer/whatthefuck/arcology/publishing/FeedModels.kt:noweb yes
<<feed-models>>

PubdateParser.kt

kotlin#+name: pubdate-parser-assembly:tangle ../src/commonMain/kotlin/computer/whatthefuck/arcology/publishing/PubdateParser.kt:noweb yes
<<pubdate-parser>>

FeedRepository.kt

kotlin#+name: feed-repo-assembly:tangle ../src/commonMain/kotlin/computer/whatthefuck/arcology/publishing/FeedRepository.kt:noweb yes
<<feed-repo-interface>>

<<feed-repo-impl>>

ArcologyFeedPlugin.kt

kotlin#+name: feed-plugin-assembly:tangle ../src/commonMain/kotlin/computer/whatthefuck/arcology/publishing/ArcologyFeedPlugin.kt:noweb yes
<<feed-plugin>>

FeedPublisher.kt

kotlin#+name: feed-publisher-assembly:tangle ../src/jvmMain/kotlin/computer/whatthefuck/arcology/publishing/FeedPublisher.kt:noweb yes
<<feed-publisher>>

PubdateParserTest.kt

kotlin#+name: pubdate-parser-test-assembly:tangle ../src/commonTest/kotlin/computer/whatthefuck/arcology/publishing/PubdateParserTest.kt:noweb yes
<<pubdate-parser-test>>

FeedRepositoryTest.kt

kotlin#+name: feed-repo-test-assembly:tangle ../src/jvmTest/kotlin/computer/whatthefuck/arcology/publishing/FeedRepositoryTest.kt:noweb yes
<<feed-repo-test>>

ArcologyFeedPluginTest.kt

kotlin#+name: feed-plugin-test-assembly:tangle ../src/commonTest/kotlin/computer/whatthefuck/arcology/publishing/ArcologyFeedPluginTest.kt:noweb yes
<<feed-plugin-test-prelude>>

<<feed-plugin-test>>

<<feed-plugin-test-end>>

FeedPublisherTest.kt

kotlin#+name: feed-publisher-test-assembly:tangle ../src/jvmTest/kotlin/computer/whatthefuck/arcology/publishing/FeedPublisherTest.kt:noweb yes
<<feed-publisher-test>>
Contents