BIDS Integration
Reading and writing Brain Imaging Data Structure datasets
Overview
TVBO reads and writes BIDS [1] datasets, so a simulation can consume an empirical connectome and publish its results where neuroimaging tooling can find them. A study is a BIDS dataset in its own right — DatasetType: "study", the type BIDS has carried since 1.10.1 — and three extension proposals reach into it: BEP034 for computational modelling [2], BEP017 for connectivity matrices, and BEP028 for provenance.
All three are still drafts. Everything below is generated from the layout record and from the code that builds the filenames, so where TVBO leaves the standard this page says which entry, and which proposal would settle it, rather than claiming a conformance it does not have.
Tutorials
| Tutorial | Description |
|---|---|
| Computational Modelling | What BEP034 specifies, and where TVBO differs |
A study is a BIDS study dataset
tvbo study init <Name> creates the dataset. Its shape comes from one record, schema/study_layout.yaml, which is also what the path resolvers, the ignore-file writers, tvbo validate study and every block on this page read — so no copy of the tree can fall out of step with the tooling.
<Study>/
dataset_description.json declares the dataset type, its name, and the BIDS version it was written against
README.md what the study does, and how to run it
CITATION.cff
<Study>.yaml the entry recipe: the one specification a run is given
.gitignore
.bidsignore
spec/ recipe fragments the entry recipe includes, each named for what it specifies: `model-<name>_dynamics.yaml`, `atlas-<name>_network.yaml`, `exp-<id>_experiment.yaml`, `ana-<name>_analysis.yaml`, `fig-<id>_figure.yaml`
code/ callables the recipe references by bare `module:` name: builders, transforms, observation and analysis functions
sourcedata/ inputs the study did not compute
README.md where each input comes from and how to obtain it
docs/ the report and everything it reads
report.qmd the report, every number computed from the run
_quarto.yml
references.bib
report.pdf rendered from `report.qmd`, so it is a product like any other
.quarto/ quarto's own cache for this project, rewritten on every render
figures/ render target for the declarative figures, each named `fig-<id>_desc-<descriptor>` -- the identifier the document prints and a camelCase descriptor -- so a figure's record, the script that draws it, its image and its caption all carry one name
scripts/ the plotting script each figure generates, kept beside what it renders
analysis/ what the study commits to and how it is checked, written down before the run rather than read off it: a preregistration for a study proposing something of its own, and for a reproduction the target values, the figure inventory and the backend comparison whose three files below carry it
notes/ the gap register and open threads, kept local so a note can be blunt
derivatives/ nested derivative datasets
tvbo/ one flat derivative dataset holding every container this study computes: `exp-<id>_model-<name>_result.h5` for a run, `ana-<name>_result.h5` for an analysis, each beside one `.yaml` sidecar: the frozen, re-runnable spec that produced it
dataset_description.json declares the derivative type, the generating tool, and the source dataset, which is the study root two levels up
logs/ run logs
.tvbo/ build root
kits/ self-contained runnable kits packaged from the spec, each with its own shards under `<kit>/shards/`, named `split-<index>`
cache/ cached intermediate results, keyed on their inputsThat tree is the study every kind starts from. --template selects a kind the record declares, which adds to it and may replace part of it: replication, for work that reproduces a paper, adds sourcedata/original_study/ with its fig_comparisons/ staging area and the A/B report wrapper; collection, for a study that nests other studies, swaps the single report for the document under docs/sections/ and the committed manuscript_results.yml the prose quotes its numbers from. A name the record does not declare is rejected rather than quietly selecting nothing.
Nothing in that tree is typed by hand twice. A resolver asks the record for a role and gets a path:
from tvbo.utils.study_layout import study_path
study_path("results", root=study_root) # <root>/derivatives/tvbo
study_path("figures", root=study_root) # <root>/docs/figuresMoving a directory is then a one-line edit to the record, and tvbo run follows it without being told.
What is tracked, and what is not
Both ignore files are generated from the record’s per-entry tracked and bids fields. The .gitignore is the study’s copyright gate: sourcedata/ holds inputs the study did not compute, so only its README is tracked and third-party or copyrighted material is documented without being redistributed.
.gitignore
# Generated by `tvbo study init` from schema/study_layout.yaml. Edit the record, not this file.
sourcedata/*
!sourcedata/README.md
docs/.quarto/
docs/figures/
docs/notes/
derivatives/*
!derivatives/tvbo/
derivatives/tvbo/*
!derivatives/tvbo/dataset_description.json
logs/
.tvbo/
docs/report.pdf.bidsignore
# Generated by `tvbo study init` from schema/study_layout.yaml. Edit the record, not this file.
spec/
/<Study>.yamlUnder --template replication the same sourcedata/ rule also covers the A/B composites that embed the original paper’s figures, because they are staged inside the original-study directory they come from, at sourcedata/original_study/fig_comparisons/. Neither file is meant to be edited: tvbo validate study fails a study whose ignore files have drifted from the record.
Where TVBO leaves the standard
| path | status | waits on |
|---|---|---|
<Study>.yaml |
unsanctioned | BEP034+, which would sanction a recipe at the study root. A named entry rather than a *.yaml glob, so a root YAML that later ought to be validated is not silently exempt. |
spec/ |
unsanctioned | BEP034+, which proposes spec/ as a study-root directory for the model and experiment definitions a run reads. |
Two entries that look like exceptions are not. derivatives/tvbo/ is a nested derivative dataset, which BIDS sanctions and which carries its own dataset_description.json naming the study root as its source. .tvbo/ is dot-prefixed, which validators skip by convention; everything in it is a pure product of the spec and reproduces from it.
Naming a spec fragment
A fragment under spec/ carries its BIDS entities from the start, so adding a second instance renames nothing: model-<name>_dynamics.yaml, atlas-<name>_network.yaml, exp-<id>_experiment.yaml, ana-<name>_analysis.yaml, fig-<id>_figure.yaml.
The suffix is the part that says what kind of thing the file is — which every TVBO file already declares in its tvbo_class envelope. Making the two the same turns the filename into a checkable restatement of the envelope, and gives BEP034+ a suffix vocabulary to propose in place of eq/param:
| suffix | declares |
|---|---|
_dynamics |
one Dynamics |
_network |
one Network |
_experiment |
one SimulationExperiment |
_analysis |
one Analysis |
_figure |
one Figure |
_study |
one SimulationStudy |
_result is not in that vocabulary. A result container is data rather than a specification, named by the grammar below, and its YAML sidecar takes the container’s name per BIDS rather than its own class’s — so the one file whose suffix and envelope legitimately disagree is not held to a rule it cannot keep.
What a run writes
tvbo run <Study>.yaml persists into the study’s own derivatives/tvbo/. -o relocates it and changes nothing else.
derivatives/tvbo/
├── dataset_description.json DatasetType: derivative, GeneratedBy: tvbo
├── exp-3_model-JansenRit_result.h5 every recorded observation, one container
├── exp-3_model-JansenRit_result.yaml the frozen, re-runnable recipe
├── exp-3_model-JansenRit_network.h5 the frozen connectome it ran on
└── ana-fcGradient_result.h5 a declared analysis's own containerOne flat directory, so a cross-reference from an analysis to an experiment resolves against a single base. exp- marks a run and ana- a derived analysis. The YAML beside each container is the only sidecar: the whole recipe rather than a curated subset of it, so the two cannot drift.
[sub-{subject}_]exp-{experiment}[_model-{model}][_desc-{description}][_split-{split}]_{suffix<result>}{extension<.h5|.yaml>}
ana-{analysis}[_desc-{description}]_{suffix<result>}{extension<.h5|.yaml>}| entity | identifies |
|---|---|
sub- |
The subject a per-subject shard ran, when a dataset fans out over a cohort. Absent for a single-network run. |
exp- |
The experiment id the run came from — the name of one entry under the study’s experiments:. |
ana- |
The name of a declared analysis, in place of exp-. A file carries one or the other, never both. |
model- |
The dynamics the run integrated, so the one fact a reader most wants to filter on is queryable rather than buried in desc-. |
desc- |
BIDS’s free-text discriminator, for two results of the same experiment that differ in nothing a named entity captures. |
split- |
The array-task index of one shard of a sweep, zero-padded. Present only until the shards are gathered. |
The .h5 is a netCDF-flavoured xarray dataset: each recorded observation is a data variable, and any swept parameters are shared coordinates. A 15,600-cell sweep is one file rather than 15,600, and xarray.open_dataset reads it without TVBO installed.
import xarray as xr
ds = xr.open_dataset("derivatives/tvbo/exp-3_model-JansenRit_result.h5")
ds["bold"].sel(G=2.1, speed=10)What a run records
The frozen sidecar beside a container is the recipe — what was asked for. Its provenance: block records what happened, which no recipe can say: the command, when it ran, on which machine, with which package versions, and the checksum of what came out. Written from the datamodel’s own PROV classes, so a record cannot drift from the metadata a run already carries.
# derivatives/tvbo/exp-3_model-Kuramoto_result.yaml — the recipe, and beneath it the run
provenance:
date_created: 2026-09-03T06:10:20+00:00
outputs: [theta] # read off the container, not predicted from the spec
digest: {sha256: {value: aa61…}} # the artifact's own checksum
activities: # BEP028's activity record: the command, the clock, what it consumed
- {iri: "tvbo:exp/Demo/exp-3", command: "tvbo run Demo.yaml", started_at: …, ended_at: …, used: […]}
environment: # the machine, and one requirement per package at the version imported
platform: macOS-26.6.2-arm64
requirements: [{name: tvbo, version: 0.5.3}, …]BEP028 spreads the same content over four files per container — prov-<label>_{act,ent,env,soft} — in a parallel prov/ tree. One Provenance on the sidecar says it once, in the file a reader already has open, and cannot fall out of step with the spec it describes: activities is the act record, environment carries env and soft together, and digest with outputs is what ent said that the spec does not.
Every field is either a pointer or read off the artifact when it is written; nothing restates a value the frozen spec already carries. That is the rule that keeps this from becoming a second, drifting copy of the recipe.
The record is written on every run, with no flag to set: the sidecar is a product of the run and never tracked, so a study cannot end up with results whose origin it did not record. It is untracked for the same reason it is truthful — it describes one machine at one moment, and committing it would assert a fact the next person’s run contradicts.
Checking a study
tvbo validate study . # or `-t replication` for the replication layoutThe command reports, as build-failing problems rather than warnings:
- a directory or file the record marks
requiredthat is missing - a
dataset_description.jsonwhoseDatasetTypeis not what the record declares - a
.gitignoreor.bidsignorethat has drifted from what the record generates - a
spec/fragment whose suffix disagrees with its owntvbo_classenvelope - a filename entity that is not in the grammar above
tvbo study layout prints any of these references on demand — tree, gitignore, bidsignore — and tvbo study layout --sync <file> rewrites the generated regions of a document in place, which is how this page and the study README stay current without either restating the record.
Reading BIDS data
from tvbo import Network
net = Network.from_bids("sub-01/connectome")Network.to_bep017 writes a connectome back out as BEP017 per-measure files, one meas-<name>_relmat.dense.tsv plus JSON sidecar per edge measure, so a connectome tvbo read or built round-trips into the form other tooling reads.
There is one export layout. A run writes derivatives/tvbo/ as described above, and that is what the rest of TVBO resolves against. An earlier exporter wrote a parallel tree of net/, ts/, eq/, coord/ and map/ directories taken from a draft no current BEP034 artifact defines; it has been removed rather than left to imply a second answer.