Arcology Engine

Arroyo Emacs Integration

Contents

Emacs Lisp
Customization
Interactive Commands
Query and Index Commands
Modes
Transient Menu
Nix Scaffolding
Emacs Package Derivation
Arroyo Emacs Module
Arroyo Epkg Override

The arroyo.el package provides Emacs integration for the arcology2 CLI. It replaces org-auto-tangle by calling out to the tangle tool when a file is saved, and indexes it into the configured arcology.db. It also provides an interactive M-x arroyo-flood command which will reindex the files and then tangle files which depend on other files' metadata (such as those querying Emacs configuration snippets to generate an init.el).

To get this in to an Emacs using the Arroyo System Flake Generator, some things have to happen:

The lisp needs to be a valid package.el package; luckily this is not so difficult.
a Nix emacs package derivation is created, this handles native compilation, etc.
the arroyo.el package is injected in to Arroyo Emacs's package-set by arroyo-flood.
An init.el snippet is made which =use-package='s in the package and provides the opportunity to configure it. This is picked up by arroyo-flood and added to the init.el
=emacs-overlay='s emacsWithPacakgesFromUsePackage reads the use-package declaration from the generated init.el, finds the package in the overidden package set, sets up load-path to find it.
When Emacs starts up, it reads the init.el, finds the native-compiled arroyo.eln in load-path and loads it in the Emacs environment with the customizations applied.

In this way, it is possible to download an org-mode file on to your computer and have it inject never before seen behavior in to your Emacs.

Emacs Lisp

emacs-lisp:tangle ../lisp/arroyo.el:comments none

;;; arroyo.el --- Arroyo literate programming integration -*- lexical-binding: t; -*-
;; Copyright (C) 2026 Ryan Rix
;; Author: Ryan Rix <code@whatthefuck.computer>
;; Version: 0.2.0
;; Package-Requires: ((emacs "28.1"))
;; Keywords: org, literate-programming, tools
;; URL: https://engine.arcology.garden

;;; Commentary:
;; Integration with arcology2go CLI for:
;; - org-auto-tangle replacement (tangle on save with indexing)
;; - M-x arroyo-flood (full system rebuild)
;; - global minor mode (arroyo-mode) with menu-bar and tool-bar integration
;; - M-x arroyo-dispatch (transient menu wrapping the arcology2 CLI)
;;
;; Only triggers an index for files with #+AUTO_TANGLE: t in their preamble.

Customization

arroyo-db-path, arroyo-cli-path, and arroyo-org-directory are all defcustom variables so the user can point them at their own org directory and database. For example, if you aren't running a fully instantiated Arroyo System yet you will want to =(setq arroyo-cli-path "nix run https://code.rix.si/rrix/arcology2go?submodules=1 --")= or so

emacs-lisp:tangle ../lisp/arroyo.el:comments none

;;; Code:

(require 'transient)

(defgroup arroyo nil
  "Arroyo literate programming system."
  :group 'org
  :prefix "arroyo-")

(defcustom arroyo-db-path (expand-file-name "~/org/arcology.db")
  "Path to arcology database."
  :type 'file
  :group 'arroyo)

(defcustom arroyo-cli-path "arcology2"
  "Path to arcology CLI binary."
  :type 'file
  :group 'arroyo)

(defcustom arroyo-org-directory (expand-file-name "~/org")
  "Root org directory for arroyo operations."
  :type 'directory
  :group 'arroyo)

Interactive Commands

The package's interactive surface is the arroyo-dispatch transient (see the Transient Menu section), but every entry is also a standalone M-x command. arroyo-tangle is a replacement for org-babel-tangle-file which indexes the file into the Arcology database after extracting the source code. arroyo-flood and arroyo-flood-all use the database to tangle files which rely on metadata extracted from other files. The query and index commands (arroyo-index-directory, arroyo-search, arroyo-stats, arroyo-keywords, arroyo-nixos-modules, arroyo-home-modules, arroyo-emacs-snippets, arroyo-emacs-epkgs) shell out to the corresponding arcology2 subcommands.

arroyo-flood runs the CLI asynchronously via async-shell-command so Emacs remains responsive during the full rebuild. arroyo-tangle runs synchronously via call-process because it's triggered on save and should complete before the user continues editing, however the database indexing is done asynchronously so beware flooding immediately after a tangle..

Tangle resolves the CLI's --repo-root from the project the org file lives in when project.el or projectile.el claim it, so org files worked on through project-switch-project (or tramp, where arroyo-org-directory would be wrong) get repo-root-relative tangle markers computed against the right root; it falls back to arroyo-org-directory when the file is not in a project.

emacs-lisp:tangle ../lisp/arroyo.el:comments none

;;;###autoload
(defun arroyo-tangle ()
  "Tangle current org file and index it into arcology.db.
Only runs if buffer has #+AUTO_TANGLE: t keyword."
  (interactive)
  (let ((org-file (buffer-file-name)))
    (when (and org-file
               (string-suffix-p ".org" org-file))
      (message "Arroyo: tangling %s..." (file-name-nondirectory org-file))
      (arroyo--tangle-file org-file)
      (when (save-excursion
               (goto-char (point-min))
               (re-search-forward "^#\\+AUTO_TANGLE:\\s-+t" nil t))
        (message "Arroyo: indexing %s..." (file-name-nondirectory org-file))
        (arroyo--index-file org-file))
      (message "Arroyo: done with %s" (file-name-nondirectory org-file)))))

(defun arroyo--project-root (org-file)
  "Return the project root for ORG-FILE, or `arroyo-org-directory'.
Uses project.el or projectile.el when the file is claimed by a
project, so that tangling files in a project directory uses that
project's root for repo-relative output paths."
  (or (when (fboundp 'project-current)
        (when-let* ((project (project-current nil (file-name-directory org-file)))
                    (root (project-root project)))
          (expand-file-name root)))
      (when (fboundp 'projectile-project-p)
        (when (projectile-project-p org-file)
          (projectile-project-root)))
      arroyo-org-directory))

(defun arroyo--tangle-file (org-file)
  "Tangle ORG-FILE using arcology CLI."
  (let* ((repo-root (arroyo--project-root org-file))
         (default-directory repo-root))
    (call-process arroyo-cli-path nil nil nil
                  "tangle" org-file
                  "--db" arroyo-db-path
                  "--repo-root" repo-root)))

(defun arroyo--index-file (org-file)
  "Index the arroyo org directory into the arcology database.
ORG-FILE is accepted for compatibility but the whole `arroyo-org-directory'
is reindexed asynchronously so cross-file metadata is consistent."
  (ignore org-file)
  (arroyo--run-async
   (list "index" arroyo-org-directory "--db" arroyo-db-path)
   "*arroyo-index*"))

(defun arroyo--cli-command (args)
  "Build a shell-quoted command string for `arroyo-cli-path' with ARGS."
  (mapconcat #'shell-quote-argument
             (cons arroyo-cli-path args)
             " "))

(defun arroyo--run-async (args buffer-name)
  "Run `arroyo-cli-path' with ARGS asynchronously, writing to BUFFER-NAME.
`default-directory' is bound to `arroyo-org-directory'."
  (let ((default-directory arroyo-org-directory)
        (cmd (arroyo--cli-command args)))
    (message "Arroyo: %s" cmd)
    (async-shell-command cmd (get-buffer-create buffer-name))))

emacs-lisp:tangle ../lisp/arroyo.el:comments none

(defun arroyo--expand-on-args (args)
  "Expand any `--on=a,b' entries in ARGS into repeated `--on=' flags.
The arcology2 `flood' command accepts `--on' repeated once per deploy
host; transient stores the value as a single comma-separated string, so
this splits it back out."
  (let (result)
    (dolist (arg args)
      (if (string-match "^\\(--on\\)=\\(.+\\)$" arg)
          (dolist (host (split-string (match-string 2 arg) "," t "\\s-+"))
            (push (concat "--on=" host) result))
        (push arg result)))
    (nreverse result)))

(defun arroyo--flood-with-args (args)
  "Run `arroyo flood' asynchronously with transient ARGS.
ARGS is a list of CLI flag strings (e.g. `--role=server', `--all-modules',
`--dry-run', `--rebuild=switch', `--on=edge,server')."
  (let ((expanded (arroyo--expand-on-args args)))
    (arroyo--run-async
     (append (list "flood"
                   "--org-dir" arroyo-org-directory
                   "--db" arroyo-db-path)
             expanded)
     "*arroyo-flood*")))

;;;###autoload
(defun arroyo-flood (role)
  "Run full arroyo rebuild pipeline for all ARROYO_TANGLE_THIS files.
With prefix arg, prompt for ROLE filter (endpoint, server, etc.)."
  (interactive
   (list (when current-prefix-arg
           (completing-read "Role: " '("endpoint" "server" "settop" "edge" "droid")))))
  (arroyo--flood-with-args
   (if role (list (concat "--role=" role)) nil)))

;;;###autoload
(defun arroyo-flood-all (role)
  "Run full arroyo rebuild pipeline for all Arroyo module files.
With prefix arg, prompt for ROLE filter (endpoint, server, etc.)."
  (interactive
   (list (when current-prefix-arg
           (completing-read "Role: " '("endpoint" "server" "settop" "edge" "droid")))))
  (arroyo--flood-with-args
   (delq nil (list "--all-modules"
                   (when role (concat "--role=" role))))))

(defun arroyo-flood-run (args)
  "Run `arroyo flood' with transient ARGS from `arroyo-flood-menu'."
  (interactive (list (transient-args 'arroyo-flood-menu)))
  (arroyo--flood-with-args args))

(defun arroyo-flood-run-all (args)
  "Run `arroyo flood --all-modules' with transient ARGS.
`--all-modules' is added unless already present in ARGS."
  (interactive (list (transient-args 'arroyo-flood-menu)))
  (arroyo--flood-with-args
   (if (or (member "--all-modules" args) (member "-a" args))
       args
     (cons "--all-modules" args))))

Query and Index Commands

The remaining arcology2 subcommands are exposed as interactive commands that shell out to the CLI and pop a buffer with the output. Long-running commands (index) run asynchronously via arroyo--run-async; the query commands (search, stats, keywords, the module listers) run synchronously with call-process into a fresh buffer so the result is available immediately. Each accepts a prefix arg to override the role filter where the underlying CLI supports one.

emacs-lisp:tangle ../lisp/arroyo.el:comments none

(defun arroyo--run-sync (args buffer-name)
  "Run `arroyo-cli-path' with ARGS synchronously into BUFFER-NAME.
`default-directory' is bound to `arroyo-org-directory'."
  (let ((default-directory arroyo-org-directory)
        (buf (get-buffer-create buffer-name)))
    (with-current-buffer buf
      (let ((inhibit-read-only t))
        (erase-buffer)))
    (apply #'call-process arroyo-cli-path nil buf nil args)
    (pop-to-buffer buf)))

;;;###autoload
(defun arroyo-index-directory (directory)
  "Index DIRECTORY into the arcology database.
Reindexes asynchronously into the `*arroyo-index*' buffer."
  (interactive
   (list (read-directory-name "Index directory: " arroyo-org-directory nil t)))
  (arroyo--run-async
   (list "index" (expand-file-name directory) "--db" arroyo-db-path)
   "*arroyo-index*"))

;;;###autoload
(defun arroyo-search (query mode limit)
  "Search indexed org files for QUERY.
MODE is one of primary, content, combined, simple.  LIMIT caps results."
  (interactive
   (let* ((query (read-string "Search: "))
          (mode (completing-read "Mode: "
                                 '("combined" "primary" "content" "simple")
                                 nil t "combined"))
          (limit (read-number "Limit: " 20)))
     (list query mode limit)))
  (arroyo--run-sync
   (list "search" query
         "--db" arroyo-db-path
         "--mode" mode
         "--limit" (number-to-string limit))
   "*arroyo-search*"))

;;;###autoload
(defun arroyo-stats ()
  "Show arcology database statistics."
  (interactive)
  (arroyo--run-sync
   (list "stats" "--db" arroyo-db-path)
   "*arroyo-stats*"))

;;;###autoload
(defun arroyo-keywords (keyword)
  "Look up Arroyo KEYWORD in the database and list matching files."
  (interactive
   (list (completing-read
          "Keyword: "
          '("ARROYO_TANGLE_THIS" "ARROYO_NIXOS_MODULE" "ARROYO_HOME_MODULE"
            "ARROYO_EMACS_MODULE" "ARROYO_HOME_EPKGS" "ARROYO_SYSTEM_ROLE"
            "ARROYO_SYSTEM_EXCLUDE" "ARROYO_MODULE_WANTS" "ARROYO_MODULE_WANTED"
            "ARROYO_INPUT" "ARROYO_OUTPUT" "ARROYO_SYSTEM_OVERLAY")
          nil t)))
  (arroyo--run-sync
   (list "keywords" keyword "--db" arroyo-db-path)
   "*arroyo-keywords*"))

(defun arroyo--list-modules (subcommand role buffer)
  "Run SUBCOMMAND (`nixos-modules' or `home-modules') with ROLE into BUFFER."
  (let ((args (delq nil (list subcommand
                              (when role (concat "--role=" role))
                              "--db" arroyo-db-path))))
    (arroyo--run-sync args buffer)))

;;;###autoload
(defun arroyo-nixos-modules (role)
  "List NixOS modules from ARROYO_NIXOS_MODULE.
With prefix arg, prompt for ROLE filter."
  (interactive
   (list (when current-prefix-arg
           (completing-read "Role: " '("endpoint" "server" "settop" "edge" "droid")))))
  (arroyo--list-modules "nixos-modules" role "*arroyo-nixos-modules*"))

;;;###autoload
(defun arroyo-home-modules (role)
  "List Home Manager modules from ARROYO_HOME_MODULE.
With prefix arg, prompt for ROLE filter."
  (interactive
   (list (when current-prefix-arg
           (completing-read "Role: " '("endpoint" "server" "settop" "edge" "droid")))))
  (arroyo--list-modules "home-modules" role "*arroyo-home-modules*"))

;;;###autoload
(defun arroyo-emacs-snippets ()
  "List Emacs snippets from ARROYO_EMACS_MODULE, topologically sorted."
  (interactive)
  (arroyo--run-sync
   (list "emacs-snippets" "--db" arroyo-db-path)
   "*arroyo-emacs-snippets*"))

;;;###autoload
(defun arroyo-emacs-epkgs ()
  "List Emacs epkg overrides from ARROYO_HOME_EPKGS."
  (interactive)
  (arroyo--run-sync
   (list "emacs-epkgs" "--db" arroyo-db-path)
   "*arroyo-emacs-epkgs*"))

;;;###autoload
(defun arroyo-set-variable (var)
  "Set an Arroyo customization variable interactively."
  (interactive
   (list (completing-read
          "Variable: "
          '("arroyo-db-path" "arroyo-cli-path" "arroyo-org-directory")
          nil t)))
  (customize-set-variable (intern var)))

;;;###autoload
(defun arroyo-customize-group ()
  "Open the `arroyo' customization group."
  (interactive)
  (customize-group 'arroyo))

Modes

arroyo-mode is a global minor mode that enables arroyo-auto-tangle-mode in every org-mode buffer and disables the legacy org-auto-tangle-mode (if it is loaded). It exposes the arcology commands in a menu-bar menu visible only in org-mode buffers, and adds an arroyo-flood button to the tool-bar of org-mode buffers.

The menu is bound to org-mode-map (not arroyo-mode-map) so that it appears only when the major mode is active, rather than in every buffer the global minor mode touches.

emacs-lisp:tangle ../lisp/arroyo.el:comments none

;;;###autoload
(define-minor-mode arroyo-auto-tangle-mode
  "Minor mode to automatically tangle and index org files on save.
Only activates for files with #+AUTO_TANGLE: t."
  :lighter " arroyo"
  :group 'arroyo
  (if arroyo-auto-tangle-mode
      (add-hook 'after-save-hook #'arroyo-tangle nil t)
    (remove-hook 'after-save-hook #'arroyo-tangle t)))
(when (fboundp 'diminish) (diminish 'arroyo-auto-tangle-mode))

emacs-lisp:tangle ../lisp/arroyo.el:comments none

(defvar arroyo-mode-map (make-sparse-keymap)
  "Keymap for `arroyo-mode'.")

(define-key arroyo-mode-map (kbd "C-c a") #'arroyo-dispatch)

(defvar arroyo-mode-menu-spec
  '("Arroyo"
    ["Dispatch…" arroyo-dispatch t]
    "--"
    ["Tangle and Index Buffer" arroyo-tangle t]
    ["Flood…" arroyo-flood t]
    ["Flood All Modules" arroyo-flood-all t])
  "Menu specification for `arroyo-mode', installed on `org-mode-map'.")

(defun arroyo--enable-in-org-buffer ()
  "Enable `arroyo-auto-tangle-mode' and disable legacy auto-tangle in current buffer."
  (arroyo-auto-tangle-mode 1)
  (when (fboundp 'org-auto-tangle-mode)
    (org-auto-tangle-mode -1)))

(defun arroyo--disable-in-org-buffer ()
  "Disable `arroyo-auto-tangle-mode' in current buffer."
  (arroyo-auto-tangle-mode -1))

;;;###autoload
(define-minor-mode arroyo-mode
  "Global minor mode for Arroyo literate programming integration.
Enables `arroyo-auto-tangle-mode' in all org-mode buffers and
disables the legacy `org-auto-tangle-mode'.  Installs a menu-bar
menu on `org-mode-map' and a tool-bar button for `arroyo-flood',
both visible only in org-mode buffers."
  :global t
  :lighter " Arroyo"
  :group 'arroyo
  :keymap arroyo-mode-map
  (if arroyo-mode
      (progn
        (add-hook 'org-mode-hook #'arroyo--enable-in-org-buffer)
        (easy-menu-define arroyo-mode-menu org-mode-map
          "Arroyo literate programming menu."
          arroyo-mode-menu-spec)
        (tool-bar-add-item "save" #'arroyo-flood 'arroyo-flood
                           :visible '(derived-mode-p 'org-mode)
                           :help "Run arroyo flood (full system rebuild)")
        (dolist (buf (buffer-list))
          (with-current-buffer buf
            (when (derived-mode-p 'org-mode)
              (arroyo--enable-in-org-buffer)))))
    (remove-hook 'org-mode-hook #'arroyo--enable-in-org-buffer)
    (when (keymapp (lookup-key org-mode-map [menu-bar arroyo-mode-menu]))
      (define-key org-mode-map [menu-bar arroyo-mode-menu] nil))
    (condition-case nil
        (tool-bar-local-item-remove 'arroyo-flood)
      (error nil))
    (dolist (buf (buffer-list))
      (with-current-buffer buf
        (when (derived-mode-p 'org-mode)
          (arroyo--disable-in-org-buffer))))))

arroyo-dispatch is the front door to the package: a transient popup that wraps the arcology2 CLI. It groups the buffer-local actions (tangle), the long-running pipeline (flood), the database queries (search, stats, keywords, the module listers), the auto-tangle toggles, and customization entry points. The flood actions pop a dedicated sub-transient, arroyo-flood-menu, which exposes every flood CLI flag as an infix so a full role-filtered, rebuild-on-hosts invocation can be assembled without leaving the popup.

The transient infixes emit the same flag strings the CLI accepts (=--role=endpoint=, --all-modules, --dry-run, --no-update, --ignore-errors, --verbose, =--rebuild=switch=, =--on=edge,server=). --on is stored as a single comma-separated value and expanded to repeated --on flags by arroyo--expand-on-args before the CLI call, since transient stores multi-values as one string. --role and --rebuild use :choices so the value is read with completion over the roles the indexer knows about and the nixos-rebuild actions the flood command supports.

arroyo-flood and arroyo-flood-all keep their original interactive signatures (prefix-arg prompts for a role) so existing key bindings and the tool-bar button continue to work; both delegate to arroyo--flood-with-args, which is also what the transient run actions call.

emacs-lisp:tangle ../lisp/arroyo.el:comments none

;;;###autoload
(transient-define-prefix arroyo-dispatch ()
  "Arroyo dispatch menu — wraps the `arcology2' CLI."
  [["Actions"
    ("t" "Tangle and index buffer"   arroyo-tangle)
    ("i" "Index org directory…"      arroyo-index-directory)
    ("s" "Search…"                   arroyo-search)
    ("f" "Flood…"                    arroyo-flood-menu)]
   ["Queries"
    ("S" "Database stats"            arroyo-stats)
    ("k" "Lookup keyword…"           arroyo-keywords)
    ("n" "NixOS modules…"            arroyo-nixos-modules)
    ("h" "Home Manager modules…"     arroyo-home-modules)
    ("e" "Emacs snippets"            arroyo-emacs-snippets)
    ("p" "Emacs epkg overrides"      arroyo-emacs-epkgs)]
   ["Toggles"
    ("a" "Auto-tangle (buffer)"      arroyo-auto-tangle-mode)
    ("A" "Arroyo mode (global)"      arroyo-mode)]
   ["Customize"
    ("v" "Set variable…"             arroyo-set-variable)
    ("g" "Customize group"           arroyo-customize-group)]])

;;;###autoload
(transient-define-prefix arroyo-flood-menu ()
  "Arroyo Flood sub-menu — assemble `arcology2 flood' flags and run.
Selected infix values are remembered between invocations and across
Emacs sessions (via `transient-history-file')."
  :remember-value '(save exit)
  [["Arguments"
    ("-r" "Role"              "--role=" :choices ("endpoint" "server" "settop" "edge" "droid"))
    ("-a" "All modules"       "--all-modules")
    ("-n" "Dry run"           "--dry-run")
    ("-u" "Skip flake update" "--no-update")
    ("-i" "Ignore errors"     "--ignore-errors")
    ("-v" "Verbose"           "--verbose")]
   ["Rebuild"
    ("-b" "Rebuild action"    "--rebuild=" :choices ("build" "switch" "test" "boot"))
    ("-o" "Deploy host(s)"    "--on=" :prompt "Hosts (comma-sep): ")]]
  [["Run"
    ("f" "Flood"              arroyo-flood-run)
    ("a" "Flood all modules"  arroyo-flood-run-all)
    ("q" "Quit"               transient-quit-one)]])

emacs-lisp:tangle ../lisp/arroyo.el:comments none

(provide 'arroyo)
;;; arroyo.el ends here

Nix Scaffolding

Emacs Package Derivation

The package is built with trivialBuild and exposed as packages.emacsPackages.arroyo in the repo flake so the Arroyo Emacs epkg override can pull it in. This follows the same callPackage pattern as default.nix (the stable CLI binary).

nix:tangle ../emacs-pkg.nix

{ pkgs ? import <nixpkgs> {} }:

pkgs.emacs.pkgs.trivialBuild {
  pname = "arroyo";
  version = "0.2.0";
  src = ./lisp/arroyo.el;
  packageRequires = [ ];
  meta = with pkgs.lib; {
    description = "Arroyo literate programming integration for arcology2go CLI";
  };
}

Arroyo Emacs Module

The loader module is tangled to ~/nix/lisp/arroyo.el so arroyo.emacs_init() picks it up in topological order (after the literate-programming module which sets up org-mode and the legacy org-auto-tangle that arroyo-mode disables). It loads the package via use-package with :ensure t, which resolves through the epkg override below, and enables the global minor mode.

emacs-lisp:tangle ~/nix/lisp/arroyo.el:mkdirp yes:comments link

(provide 'cce/arroyo)
(use-package arroyo
  :ensure t
  :config
  (arroyo-mode 1))

Arroyo Epkg Override

The override is inlined in to the override epkgs: epkgs // rec { ... }= block of Arroyo Emacs's ~/nix/pkgs/emacs.nix, making epkgs.arroyo resolve so (use-package arroyo :ensure t) finds it. It references the flake output declared in the package derivation and exposed via the repo flake's packages.emacsPackages.arroyo. pkgs.stdenv.hostPlatform.system is used to index the per-system flake output to avoid the pkgs.system deprecation warning.

nix:tangle ~/nix/overrides/arroyo.nix:mkdirp yes

arroyo = inputs.arcology2go.emacsPackages.${pkgs.stdenv.hostPlatform.system}.default;