How this documentation is built

The generated maps, the Quarto filters and the linters that keep pages honest

Where the sidebar, the phase badges, the running-example pointers and the use-case tags come from, and which checks run in CI.

Most of what surrounds a page here is derived rather than typed. The phase badge in its corner, the pointer into the running example, the list of published studies at its foot: all three come from generated maps, so a page cannot claim a phase it is not in or a study that stopped using its feature. This page is the map of that machinery, for anyone changing it.

One sidebar, one spine

_toc.yml is the single sidebar. Its top level is a flat list of chapter headings — GET STARTED, SIMULATION EXPERIMENTS WITH TVB-O, EXAMPLES & USE-CASES, AGENTIC CODING, INTEROPERABILITY, REFERENCE, DEVELOPER — each a text-only row with no href:, and the pages of a chapter are the rows that follow it.

SIMULATION EXPERIMENTS WITH TVB-O is the exception, and the reason the spine reads as one chapter rather than as a continuation of the one above it: the five numbered phases beneath it are section: entries, so each nests its own pages and collapses with the rest of the sidebar at collapse-level: 1. Quarto expands whichever section holds the current page. The spine’s heading is declared in _static/phases.yml under spine:, which is how scripts/build_phase_map.py knows that row introduces the phases rather than owning pages of its own.

Three regions of the file are script-owned and marked as such:

Region Written by
# BEGIN:replication-autogen scripts/update_toc_replication.py
# BEGIN:datamodel-autogen scripts/build_datamodel_pages.py
# BEGIN:api-autogen quartodoc, via scripts/prerender.sh

Edit outside those markers freely. Inside them, change the script.

The tree is the sidebar

Every hand-written page lives under the part that owns it, so ls and the sidebar tell the same story:

Directory Part
index.qmd, installation.md, GettingStarted.qmd, reference.qmd GET STARTED, and the reference landing page
1-explore/5-share/ the five numbered phases, each with the part’s overview page as its index.qmd
examples/, Replication/ EXAMPLES & USE-CASES
agents/, Interoperability/, CLI/, Developer/ AGENTIC CODING, INTEROPERABILITY, REFERENCE, DEVELOPER
api/, datamodel/ REFERENCE, generated

A sidebar section: with an href becomes a directory whose index.qmd is that href’s page, which is why 2-specify/Networks/index.qmd holds what the sidebar calls Networks & connectomes.

Each part declares the directories it owns in _static/phases.yml, and scripts/build_phase_map.py names every page filed outside them. Four pages are deliberate exceptions the declaration records: the SPECIFY, GENERATE and SHARE parts each link into Interoperability/, because that chapter’s 26 pages are one tree and splitting BIDS or NeuroML across four phase directories would cost more than the misalignment.

Moving a page means moving three things together: the page, its _freeze/<path>/ results, and its directory’s .jupyter_cache. Add the old URL to the page’s aliases: so existing links keep resolving; _freeze/<path>/execute-results/html.json stores an md5 of the source, so repointing that hash after an edit is what keeps a moved page from re-executing.

The phase badge

_static/phases.yml declares the five numbered phases and the six unnumbered chapters, each with a name, an icon and a colour. It is the only place any of those are written down, so swapping an icon set is one edit here. A colour is named, never written: palette.N for a categorical hue or a role such as base, resolved through the palette TVB-O ships.

scripts/build_phase_map.py walks _toc.yml against that file and writes two generated artifacts, both gitignored:

  • _static/phase_map.lua, one entry per page: its phase key, number, name, icon, colour and the section it sits in.
  • _static/phases.css, the --phase-1--phase-5 custom properties the hero pipeline and the badge both read.

filters/phase-badge.lua stamps the badge from that map. A page may override it with phase: in its own frontmatter, naming either the number or the key from phases.yml, and phase-section: sets the text after the separator. phase: false suppresses the badge entirely. A page that is in no PART and declares no phase: produces a build warning rather than a silently unbadged page, which is how the 38 NeuroML example pages were caught when the gallery took their place in the sidebar.

One palette

tvbo/plot/palette.yaml, inside the package, is where the colours are decided — for the figures a study renders, for the manuscript’s, and for these pages. It is a Palette in the figure spec, so tvbo validate schema checks it like any other TVB-O document, and tvbo.plot.palette is the one reader: a figure gets its roles by calling it, and scripts/build_palette.py writes the same five roles and five hues into _static/palette.css as --palette-*.

styles.css maps the docs’ own semantic names onto those in a single :root block and mixes every surface tint and hover state from them with color-mix, so a page cannot introduce a colour the palette has not approved, and a figure sitting on that page cannot disagree with it. A checkout of the manuscript is reported, not read: if its figures/style/palette.yaml has drifted, the pre-render says so and names the packaged file as the one to copy over it.

Overview cards

An overview page lists the pages beneath it as a pipe table wrapped in ::: {.cards}, and filters/cards.lua renders that table as a card grid. The table stays the editable form — a link, a sentence, and an optional cover — while the reader gets covers, titles and hover affordances.

Column What it holds
1 The link. Its text is the card title and its target is where the card goes; a row with no link is skipped.
2 The sentence under the title.
3 The cover, optional: a markdown image, or a Font Awesome class such as fa-solid fa-plug.

A row that names no cover falls back to the icon of the phase its target sits in, so a grid always has covers. Colours come from the same phase_map.lua the badge reads, which is why a card pointing out of the current phase wears that other phase’s colour.

Cover images come from _static/thumbs/, written by scripts/build_thumbnails.py from the declaration in _static/thumbs.yml: a page path, and the label of the figure on that page that stands in for it. The figure is read out of _freeze, so a page whose label has changed or that has never been executed in the working tree produces a build note and a card that falls back to its phase icon.

A table that compares several dimensions rather than indexing pages — the integration matrix and the bifurcation plan — stays a table, because a card carries one sentence and would drop the other columns.

The running-example thread

_static/running_example.yml maps each stage of the running example onto the pages whose block that stage adds. filters/running-example.lua reads it from both ends: the pointer strip it inserts at the top of each listed page, and the stage that pointer links to. Because both come from one file, a pointer cannot outlive its stage. A page opts out with running-example: false.

Use-case tags

_static/usecase_slots.yml says which page owns which specification slot. scripts/build_usecase_tags.py reads the hand-authored recipes of the use-case corpus, records the slots each study actually writes, and emits two committed artifacts:

  • _static/usecase_tags.lua, which filters/usecase-tags.lua renders as the Used in published replications block at the foot of an owning page.
  • _static/usecase_studies.yml, the catalogue Replication studies renders.

The corpus lives in a separate repository, so this is a maintainer step rather than part of the build:

python docs/scripts/build_usecase_tags.py --corpus ~/projects/TVB-O/tvbo-manuscript/use-cases

Nothing in that script counts a study’s experiments or figures. The recipes reach those through !include, YAML anchors and nested blocks, so a text scan gets it wrong in both directions, and a wrong number in a catalogue is worse than no number.

The linters

Five checks run over the documentation, four of them in CI.

Check What it catches In CI
slopfmt.py hard-wrapped paragraphs and stacked comment blocks yes
scripts/check_pages.py em-dash density, placeholder alt text, sequential-step tabsets, the templated index skeleton yes
scripts/check_cli_examples.py a documented tvbo command, sub-command or long option the CLI does not have yes
scripts/check_render_coverage.py a sidebar link matched by no project.render pattern, or pointing at a file that is gone yes
scripts/check_links.py an absolute link the live web does not resolve no
scripts/unwrap_prose.py the same hard wraps as slopfmt, but rewrites them no

check_pages.py counts em-dashes in running prose only. Fenced code, table rows and headings are excluded, and so is the **Label** — the gloss idiom a definition list is written in, because none of those is a sentence reaching for a connector. What is left is mid-sentence use, which is what the budget is about.

check_cli_examples.py resolves each invocation against the installed Typer app rather than a list. It reports how many it resolved and how many it skipped as templated, so a page full of <placeholder> commands cannot read as verified coverage.

unwrap_prose.py reuses slopfmt’s detector rather than reimplementing it, and refuses to write when the whitespace-normalised text would change or when the file has unbalanced code fences. Both guards exist because an earlier pass joined a heading to the paragraph below it and mangled code in two files whose fences were already unbalanced.

check_render_coverage.py exists because the sidebar and the render set are written in different files and Quarto never compares them: it builds what the globs resolve to and says nothing about what the sidebar asks for. A page can therefore be linked, present on disk, and built by nothing — which shows up only as a 404 on the deployed site, since a local preview renders on demand and hides it. Its first run found two: GettingStarted.qmd, the target of the front page’s own Get started button, and the BIDS comparison page, a .md under a directory whose glob only took .qmd.

check_links.py stays out of CI deliberately: it needs the network, and a third-party site being briefly down is not a reason to fail a build. Its first run found nine dead links, including the site’s own repo-url, which had been generating a broken Edit this page button on every page. A 403 from it usually means the publisher blocks automated requests rather than that the link is dead; check such a DOI against api.crossref.org/works/<doi> before changing it.

make docs-lint      # the four gates, plus the formatter
make docs-unwrap    # rewrite the hard wraps
make docs-links     # resolve every absolute link (network)

Rendering in parts

The site is three things with very different costs: the hand-written guide, the API reference generated by quartodoc, and the schema reference generated from the LinkML model. Together they are 1239 pages, of which the guide is 162, and regenerating the two references costs more than rendering everything else put together. Building all of it to check one paragraph is the wrong loop.

Command Builds Pages
make docs-render-guide the hand-written guide 162
make docs-render-api the API reference 182
make docs-render-datamodel the schema reference 895
make docs-render everything, for deploy 1239

The two references are self-contained subtrees, so Quarto narrows them by path: quarto render api renders that directory and nothing else. The guide is the complement of both, which a path cannot express, so it is a Quarto profile — _quarto-guide.yml subtracts api/** and datamodel/**. That asymmetry is forced: project.render entries append across profiles, so a profile can subtract from the base list but never replace it, and excluding everything then re-including one directory resolves to an empty set.

scripts/prerender.sh reads QUARTO_PROFILE and builds only the references the render can reach. Their _toc.yml regions are committed, so a skipped reference still appears in the sidebar with its full contents — a narrow render produces a partial site, not a broken one. Use make docs-render before publishing.

make docs-preview-guide is the authoring loop: it renders the guide once, then serves it with the pre-render skipped entirely.

Executing the pages

Cells run under a kernelspec pinned by the docs-kernel Makefile target. Quarto’s jupyter: python3 names a kernel whose argv is a bare python, so PATH decides which interpreter runs the cells and QUARTO_PYTHON does not change that. Without the pinned kernel a stale released tvbo from another virtualenv can win, which surfaces as constructor errors on pages that are perfectly correct.

make docs-kernel     # write the pinned kernelspec
make docs-preview    # serve with it
make docs-render     # full build with it

execute: {cache: true, freeze: auto} re-executes a page only when its source changes. _freeze/ stays out of the repository: it is 70 MB of images and JSON that churns on every content edit, and the release build renders cold on purpose, so a shared cache would buy speed at the cost of the thing that build exists to prove.

Generated, not typed

Three claims that used to rot are now read from the software at render time: the CLI command tree from the Typer app, the model count from Dynamics.list_db(), and the backend capability table from tvbo.cli._backends.BACKENDS. Prefer that pattern over a hand-kept list whenever the software already knows the answer.