Architecture

StarSpec is a compiler. The same .mdoc source files are processed by three independent passes, each targeting a different audience. No separate build step — pnpm compile runs all three.

Conceptual Model

graph Conceptual Model

Gallery Source ⛶

Diagram Syntax Error

graph TD
Manifest["📋 manifest\n─────────────\n{value}\n{principle}\n{goal}\n{requirement}"]
Role["👤 role\n─────────────\n{role}"]
Domain["📦 domain\n─────────────\n{glossary} / {term}\n{model}\n{api} / {action} / {event}\n{policy}"]
Feature["✨ feature\n─────────────\n{feature}\n{requirement}\n{api} / {action} / {event}\n{criteria} / {criterion}"]
Flow["🔀 flow\n─────────────\n{flow}\n{precondition}\n{step} / {branch} / {path}\n{postcondition}"]
Story["📖 story\n─────────────\n{story}"]
Milestone["🏁 milestone\n─────────────\n{milestone}\n{includes}"]
Manifest -. "guides" .-> Feature
Manifest -. "guides" .-> Domain
Role -- "performs" --> Feature
Feature -- "belongs to" --> Domain
Feature -- "detailed by" --> Flow
Story -- "narrates" --> Role
Story -- "references" --> Feature
Milestone -- "includes" --> Feature

Compilation Pipeline

ASCII diagram of the three-pass pipeline

architecturepipelinediagram compilation-pipeline-ascii

content/**/*.mdoc
       │
       ▼
  ┌──────────────────────────────────────┐
  │           StarSpec Compiler           │
  │                                      │
  │  ┌─────────────┐  ┌───────────────┐  │
  │  │  agent pass │  │  public pass  │  │
  │  │ (Data Ext.) │  │ (6-phase MDX) │  │
  │  └──────┬──────┘  └──────┬────────┘  │
  │         │                │           │
  │  ┌──────────────┐        │           │
  │  │instructions  │        │           │
  │  │    pass      │        │           │
  │  └──────┬───────┘        │           │
  └─────────┼────────────────┼───────────┘
            │                │
            ▼                ▼
   dist/bundles/        src/content/docs/
   {type}/{slug}.toon   {type}/{slug}.mdx
   starspec/**/*.toon   sidebar.json
            │           summary.json
            │                │
            ▼                ▼
       MCP server      Astro + Starlight
       POST /mcp       → public docs site
       POST /api/call

Pass	Input	Output	Purpose
agent	content/**/*.mdoc	dist/bundles/{type}/{slug}.toon	Extract structured data for AI consumption
public	content/**/*.mdoc	src/content/docs/{type}/{slug}.md	Transform spec to human-readable MDX
instructions	instructions/**/*.mdoc	dist/bundles/starspec/**/*.toon	Bundle engineering docs for agents

Repository Layout

starspec/
├── content/                    ← spec source files (app-specific)
│   ├── roles/        *.role.mdoc
│   ├── domains/      *.domain.mdoc
│   ├── features/     *.feature.mdoc
│   ├── flows/        *.flow.mdoc
│   ├── milestones/   *.milestone.mdoc
│   ├── stories/      *.story.mdoc
│   ├── manifests/    *.manifest.mdoc
│   ├── surfaces/     *.surface.mdoc
│   ├── tours/        *.tour.mdoc
│   └── articles/     *.article.mdoc
├── instructions/               ← agent how-to guides (engine-owned)
│   ├── tag/          *.tag.mdoc (tag reference docs)
│   └── fragments/    reusable content fragments
├── packages/
│   └── compiler/
│       └── src/
│           ├── core/           ← SOLID Domain Model (19 modules + rendering/ + tags/)
│           ├── processors/     ← Tag processors (31 files + aggregates/)
│           ├── passes/         ← Compilation passes (agent/, public/, shared/, instructions.ts)
│           ├── utils/          ← Shared utilities (api, bibtex, dbml, markdoc)
│           └── cli.ts          ← CLI entry point
├── src/
│   ├── components/             ← Unified Astro components
│   ├── content/docs/           ← generated MDX (do not edit)
│   └── styles/custom.css
├── public/
│   └── diagrams/               ← D2-rendered flow SVGs
├── astro.config.ts
└── content/
    └── starspec.config.json    ← optional project config (mode, contentDir, outDir)

Domain Model (SOLID)

StarSpec uses a robust domain model in packages/compiler/src/core/ to encapsulate logic and ensure type safety across passes.

Core Classes

• Entity: Centralized management of API prefixes (action:, event:, etc.), metadata (labels, colors), and normalization.
• Tag: Abstract base class for all Markdoc tags. Constructor takes (node: Node). Provides type guards (Tag.is), scope detection (isVisible, isPublic, isAgent), and child traversal (findTypedChildren, findTypedChild). Not generic.
• ContentBuilder: Fluent builder for assembling MDX page output. All document processors and aggregate processors use it instead of manual sections.push() + join(). Handles frontmatter, TLDR extraction, prose buffering with .ss-prose wrapping, column layout (ss-columns), headings, sections, and blocks. Constructor accepts (title, { columns? }).
• SidebarBuilder: Fluent builder for assembling sidebar.json. Replaces the imperative section assembly in sidebarPhase(). Accepts a CompileMode ('full' | 'journal') and gates sections accordingly. In journal mode, only Journal-section items are included and promoted to top-level (no wrapping group). Methods: addProcessorEntries(), addProductAggregates(), addImplementationAggregates(), addReferenceItems(), addJournalAggregates(), addChangelog(), addInstructionsGroup(), build().
• Document: Registry of document types (feature, domain, role, etc.) that handles directory and href mapping logic (toDir, toHref).
• Markup: Centralized escaping for MDX (handling curly braces and self-closing tags) and TOON bundles.
• ObjectPath: Safe recursive property manipulation — ObjectPath.push(bundle, dotPath, data) is how bundlePath inserts data into agent bundles.
• AstWalker: Generic AST walker shared by agent and public passes. Takes an array of NodeVisitor instances. Multi-visitor descent: all matching visitors run before deciding whether to descend into children.
• DiagnosticsCollector: Per-walk diagnostic accumulator used by AstWalker — collects issues during a single file’s traversal.
• IndexRegistry: General-purpose index container for cross-document lookups.
• TagIndex: Tag data index — maps bare tag id to parents, children, and properties. Built from *.tag.mdoc documents.
• CompileContext: Context container for per-pass state.
• FoldableTag: Base for collapsible tag UI rendering.
• DocRef: Document cross-reference utilities.
• scope (core/scope.ts): propagateScope(), isScoped(node, target), isPublicScoped(), isAgentScoped(). Re-exported via utils/markdoc.ts.

Rich Tag Models (core/tags/)

Complex tags are implemented as dedicated classes in core/tags/ (27 files). Each extends the base Tag class and encapsulates:

• Internal Validation: Ensuring sub-tags are correctly placed and formatted.
• Tree Traversal: Encapsulating complex lookahead logic (e.g., Step.getBranch() to identify sibling branches).
• Extraction & Rendering Logic: Moving pass-specific logic out of the entry points and into the models.

Models include: Api, Blueprint, Carousel, Changeset, Citation, DataModel, Diagram, Domain, Example, Explanation, Feature, Flow, Glossary, Manifest, Milestone, Policy, Requirement, Role, Rule, Setting, Step, Story, Surface, TagDoc, Tldr, Tour. Tag name constants live in names.ts (TagNames enum).

Rendering Utilities (core/rendering/)

Rendering logic extracted from passes/public/ into reusable modules:

• api.ts — renderApiSection(), renderApiEntry()
• blocks.ts — Block-level rendering (requirements, criteria, etc.)
• components.ts — Component and blueprint rendering
• document.ts — frontmatter(), document-level MDX utilities
• flow.ts — D2 flow rendering + renderD2ToFile()
• interactions.ts — Clickable / interaction rendering
• policies.ts — renderPoliciesSection()
• surface.ts — Surface / wireframe rendering
• utils.ts — TYPE_LABELS, typeToDir, idToHref, qnameToHref

Configuration

StarSpec reads an optional content/starspec.config.json from the project root. All fields are optional:

\{
  "contentDir": "content",
  "outDir": "public/bundles",
  "mode": "full"
\}

Field	Type	Default	Description
contentDir	string	content	Path to .mdoc source files
outDir	string	public/bundles	Path for TOON bundle output
mode	"full" | "journal"	"full"	Compilation mode (see below)

CLI flags override config file values: --content=, --out=, --mode=.

Compilation modes

Mode	Parses	Aggregates	Passes	Sidebar
full	All content/**/*.mdoc	All 24 aggregate processors	agent → instructions → public	Product, Journal, Reference, Specification, Implementation, Changelog, StarSpec
journal	Only articles/**/*.article.mdoc	Journal-relevant only (quotes, atoms, bibliography, article-tags, diagrams, examples, markers)	public only	Journal items promoted to top-level (no section wrapper)

In journal mode the agent pass, instructions pass, MCP bundle generation, and status page are all skipped. The Astro site hides spec-only UI chrome (MCP status indicator, viewport selector, wireframe theme selector) via the virtual:starspec/config Vite module.

Astro integration

astro.config.ts reads content/starspec.config.json and exposes it to components via a Vite virtual module:

import \{ isJournalMode \} from 'virtual:starspec/config';

Components use this to conditionally render spec-only UI elements. The sidebar in astro.config.ts also gates MCP-Server and System Status entries on !isJournalMode.

Compiler internals

AST walking — AstWalker + NodeVisitor

Both the agent pass and the public pass traverse the Markdoc AST using AstWalker. A visitor implements:

interface NodeVisitor {
  matches(node: Markdoc.Node): boolean;
  visit(node: Markdoc.Node, ctx: WalkContext): VisitResult; // 'continue' | 'skip-children'
}

The walker calls every matching visitor on a node before descending. If any visitor returns 'skip-children', the walker skips that node’s children for all visitors. This is the mechanism behind isContainer: a non-container processor returns 'skip-children' to prevent double-processing of child tags.

Scope propagation

Before each render pass, the compiler walks the full AST and propagates scope top-down via propagateScope() in core/scope.ts:

TypeScript propagateScope implementation

architecturescopetypescript scope-propagation-function

function propagateScope(node: Markdoc.Node, inherited: string | null): void {
  const resolved = (node.attributes?.scope as string | undefined) ?? inherited;
  if (node.attributes) node.attributes.scope = resolved;
  for (const child of node.children ?? []) {
    propagateScope(child, resolved);
  }
}

Scope is checked with isScoped(node, 'agent') or isScoped(node, 'public') (shorthand: isAgentScoped() / isPublicScoped()). A node missing scope inherits from its parent. A child can narrow or widen scope by declaring its own scope attribute.

Validation

All extracted data is validated against extract schemas derived from Tag class schemas (see validation.ts). Tag classes in core/tags/ define the authoritative attribute schema; extract schemas extend these with computed fields (text, body, content). Validation errors are collected by a ValidationTracker and logged with file and path context. The pass continues even when validation fails, using the raw data, so a single bad tag doesn’t abort the whole compile. Per-walk diagnostics are also collected by DiagnosticsCollector instances passed through WalkContext.

Agent pass (passes/agent/)

Reads content/, parses the Markdoc AST, and extracts structured data into TOON bundles. Uses AstWalker with a NodeVisitor that iterates all registered processors. Bundle placement is determined by the processor’s declarative properties:

1. merge() — for root processors that own the bundle shape
2. bundlePath — for leaf processors that push items into a specific array path
3. promotesIdentity — promotes tag id/title to the bundle root

After processing all files, the agent pass builds aggregate bundles (API, glossary, components, policies, etc.) using processor metadata (aggregatePath, aggregateIdKey).

Public pass (passes/public/) — 6-phase pipeline

The public pass is organized as a sequential pipeline of six phases:

Phase	Module	Purpose
1. Parse	phases/parse.ts	Parse all .mdoc files, resolve frontmatter
2. Extract	phases/extract.ts	Run processor extract() on each document
3. Build Index	phases/build-index.ts	Build cross-document indexes (feature↔domain, feature↔design, role↔feature, action↔surface, etc.)
4. Render	phases/render.ts	Run processor render(), write MDX pages
5. Aggregate	phases/aggregate.ts	Run AggregateProcessor instances, write aggregate pages
6. Sidebar	phases/sidebar.ts	Generate Starlight sidebar configuration from processor sidebar declarations

Each phase returns a typed result consumed by the next. Shared types live in phases/types.ts.

Sidebar grouping — SidebarBuilder

The sidebar is assembled by SidebarBuilder (core/SidebarBuilder.ts) in sidebarPhase(). Each processor declares a sidebar object that controls where and how its entries appear:

Field	Purpose
section	Top-level group: Product, Specification, Implementation, Journal
group	Sub-group label (e.g. Stories, Features, Flows)
weight	Sort order within the section (lower = higher)
collapsed	Whether the group starts collapsed in the sidebar
groupByDomain	When true, entries are further nested by domain into sub-groups

When groupByDomain is true, each entry’s domain is derived from (in order of precedence):

1. The frontmatter domain field (used by articles)
2. The first domain/ reference in the frontmatter context array (used by features, flows, surfaces)

Entries sharing a domain are grouped under the domain’s display title (from the corresponding {% domain %} document). Entries without a domain appear ungrouped at the end.

Example sidebar structure for Features with groupByDomain: true:

Specification
  └─ Features (collapsed)
       ├─ Conversation (collapsed)
       │    ├─ Structured Formats
       │    └─ ...
       ├─ Memory & Reflection (collapsed)
       │    ├─ Export Data
       │    └─ ...
       └─ Speech-To-Text (collapsed)
            └─ Transcription

To enable domain grouping for a document type, add domain/ references to the frontmatter context array:

---
type: feature
id: export-data
title: Export Data
status: draft
context: [domain/memory-reflection, role/user]
---

Aggregate pages — AggregateProcessor

Cross-document aggregate pages (glossary, API reference, examples gallery, roadmap, etc.) are produced by dedicated AggregateProcessor classes in processors/aggregates/ (22 files). Each implements:

interface AggregateProcessor<TEntry> {
  readonly id: string;
  readonly outputPath?: string;
  collect(acc, context): TEntry[];
  prepare?(entries: TEntry[]): TEntry[];
  render?(entries: TEntry[], context): string;
  write?(entries: TEntry[], context, docsDir: string): void;
  sidebarEntry?(entries: TEntry[]): AggregateSidebarEntry | null;
}

The aggregatePhase calls collect() → prepare() → render() (or write()) → sidebarEntry() for each registered processor. Processors that need multiple output files (e.g. showroom) implement write() instead of render().

Shared indexes (passes/shared/)

passes/shared/indexes.ts builds indexes used by both the agent and public passes: bundleCache, tagDataIndex, policyIndex, apiNameIndex, eventNames, operationThrowsIndex. These are built once from all content files and passed into the CompilerContext.

Instructions pass (passes/instructions.ts)

Reads instructions/ and writes dist/bundles/starspec/**/*.toon. These are engine meta-documents: design loops, tag schemas, and examples that agents can query. Resolves {% include %} directives to inline fragment content.

linkadding-a-taglinktag > domainlinktag > feature