Linked experiments in a SimulationStudy

The unit a paper is written from

A study is its experiments, the analyses that reduce them, and the figures that read both, as one object, one file and one run.

Part of the running example, where stage 6 adds several experiments, their analyses and their figures.

A SimulationExperiment answers one question. Real work asks several, and the answers have to be compared: two coupling strengths, eleven driving frequencies, a model against its control. A SimulationStudy is the object that holds them together, and it holds three kinds of thing rather than one.

SimulationStudy
  experiments  →  the runs
  analyses     →  what reduces the runs to numbers
  figures      →  what reads the runs and the analyses

That third field is what closes the loop. A study is not a folder of simulations with a plotting script beside it; the figures are part of the specification, so the thing that produced a panel is recoverable from the same file that produced the data underneath it.

A study with two experiments

The two runs differ in one parameter, so everything else is written once and referenced. &dyn names a value and *dyn reuses it, plain YAML covered in Writing TVB-O YAML.

Write the study file 
from pathlib import Path
import tempfile

STUDY = """
key: MiniStudy
label: "A two-experiment study"
experiments:
  - id: 1
    label: "Weak coupling"
    dynamics: &dyn {iri: "tvbo:Generic2dOscillator"}
    network: &net
      number_of_nodes: 4
      coupling:
        c_glob: {iri: "tvbo:Linear", delayed: false, incoming_states: [V]}
    integration: &integ {method: Heun, step_size: 0.1, duration: 300.0}
  - id: 2
    label: "Strong coupling"
    dynamics: *dyn
    network:
      <<: *net
      coupling:
        c_glob: {iri: "tvbo:Linear", delayed: false, incoming_states: [V], parameters: {a: 0.5}}
    integration: *integ
"""

root = Path(tempfile.mkdtemp())
spec = root / "MiniStudy.yaml"
spec.write_text(STUDY)
580

Load it and the experiments are addressable by the id they declare:

from tvbo import SimulationStudy

study = SimulationStudy.from_file(spec)
print(study.label)
print("experiments:", study.experiment_ids())
print("id 2 is:", study.get_experiment(2).label)
A two-experiment study
experiments: [1, 2]
id 2 is: Strong coupling

Running it

Running a study runs every experiment, then the analyses, then the figures. Both routes below do the same work.

result = study.run(root=root, figures=False)
print(type(result).__name__, "->", list(result))
INFO [tvbo.run] [+0s] STEP 1: Running simulation...
INFO [tvbo.run] [+0s]   Simulation period: 300.0 ms, dt: 0.1 ms
INFO [tvbo.run] [+0s]   Simulation complete.
INFO [tvbo.run] [+0s] Experiment complete.
INFO [tvbo.cli] done: ExperimentResult
INFO [tvbo.cli] wrote ['/private/var/folders/ym/9kw1g21j1nd7kwfn8c0z3st40000gn/T/tmp1l1bse5k/derivatives/tvbo/exp-1_model-Generic2dOscillator_result.h5', '/private/var/folders/ym/9kw1g21j1nd7kwfn8c0z3st40000gn/T/tmp1l1bse5k/derivatives/tvbo/exp-1_model-Generic2dOscillator_result.yaml', '/private/var/folders/ym/9kw1g21j1nd7kwfn8c0z3st40000gn/T/tmp1l1bse5k/derivatives/tvbo/dataset_description.json']
INFO [tvbo.cli] running experiment: Strong coupling
INFO [tvbo.run] [+0s] STEP 1: Running simulation...
INFO [tvbo.run] [+0s]   Simulation period: 300.0 ms, dt: 0.1 ms
INFO [tvbo.run] [+0s]   Simulation complete.
INFO [tvbo.run] [+0s] Experiment complete.
INFO [tvbo.cli] done: ExperimentResult
INFO [tvbo.cli] wrote ['/private/var/folders/ym/9kw1g21j1nd7kwfn8c0z3st40000gn/T/tmp1l1bse5k/derivatives/tvbo/exp-2_model-Generic2dOscillator_result.h5', '/private/var/folders/ym/9kw1g21j1nd7kwfn8c0z3st40000gn/T/tmp1l1bse5k/derivatives/tvbo/exp-2_model-Generic2dOscillator_result.yaml']
StudyResult -> ['exp-1', 'exp-2']
tvbo run MiniStudy.yaml

What comes back is keyed by experiment, and each value is an xarray.DataTree, the same container a single experiment returns, so every axis is selected by name rather than by position:

V = result["exp-1"].integration.sel(variable="V")
print("nodes:", V.sizes["node"], "| timepoints:", V.sizes["time"])
nodes: 4 | timepoints: 3000

The run also wrote a BIDS derivative tree beside the recipe. Each container has a .yaml sidecar holding the frozen, re-runnable spec that produced it, so a result can never drift away from the specification it came from:

for path in sorted((root / "derivatives" / "tvbo").iterdir()):
    print(path.name)
dataset_description.json
exp-1_model-Generic2dOscillator_network.yaml
exp-1_model-Generic2dOscillator_result.h5
exp-1_model-Generic2dOscillator_result.yaml
exp-2_model-Generic2dOscillator_network.yaml
exp-2_model-Generic2dOscillator_result.h5
exp-2_model-Generic2dOscillator_result.yaml

That layout is not incidental: it is the study directory, which tvbo study init scaffolds and tvbo validate study checks.

The rest of the object

Two experiments and no analyses is the smallest useful study. The fields below are what a real one adds, and each is documented where it is specified rather than here.

Field What it holds
experiments The runs. Each is a full SimulationExperiment.
analyses Reductions over results, each persisted as its own container and so bindable by a figure or by a later analysis exactly like a run.
figures Declarative figure specs that read the containers above.
code_source Where the callables the recipe names by bare module: live.
requires Packages this study’s own code needs beyond tvbo, with the version it was reproduced against.
workflow Study-wide defaults for rendering sweeps into Slurm, Snakemake or Nextflow jobs.
sample The subjects or cohort the study runs over.
members Other studies this one aggregates, making it a study of studies.
results The named numbers the prose cites, each bound to a computed value rather than typed into the text.
archive How to package the run as a COMBINE/OMEX archive.

One result reads another

An analysis or a figure names the container it consumes with used:, and that binding is also the provenance edge: the record of which run a number came from.

analyses:
  - name: coupling_contrast
    label: "Synchrony gained by raising the coupling"
    callable: {name: contrast, module: mini_analysis}
    arguments:
      weak:   {used: {experiment: 1, output: V}}
      strong: {used: {experiment: 2, output: V}}

Because an analysis is persisted like a run, a figure binds to it with used: {analysis: coupling_contrast} exactly as it binds to used: {experiment: 1}, and a second analysis can consume the first. Nothing in the chain is a script that has to be run in the right order by hand, because the used: edges are the order.

Analyses covers this in full, and calling your own code covers where module: mini_analysis is found.

Where to go next