A training cohort with a study-level split#

A model trained on spatial data should be tested on data from experiments it has not seen. If sections from the same experiment end up in both the training and the test fold, the test score measures how well the model recognises that experiment, not how well it generalises. The notebook builds a small cohort for training, splits it so that each experiment stays in one fold, reads one table per fold and records the split so that it can be reproduced.

It uses the steps from the tutorials, mainly 3. Cohorts and manifests, and adds split() and the TableAdapter interface. The cohort is small (five Visium sections of human pancreatic tumours, of which only the tables are read), so it runs on a laptop.

import tempfile
from pathlib import Path
from typing import Any

import sddb
from sddb import SplitManifest, SpatialDataCohort, TableAdapter

cat = sddb.Catalog(version="2026-10-28")
work = Path(tempfile.mkdtemp())  # the files this notebook writes go here

The cohort#

Five Visium sections of human pancreatic ductal adenocarcinoma from three GEO series. The collections column names the series of each section.

cohort = cat.cohort(
    uids=[
        "qEP9pd4LlyKeufe20000",
        "URKDREjVhAsRwwF10000",
        "Mc06lamGrOTYHhNF0000",
        "0877Bxvk1AFUpoD90000",
        "StreGVL1Uyj2EeoM0000",
    ]
)
cohort.to_df()[["uid", "study_id", "collections", "n_obs", "n_features"]]

	uid	study_id	collections	n_obs	n_features
0	qEP9pd4LlyKeufe20000	geo:GSM8915497	[geo-gse294669]	1142	17943
1	0877Bxvk1AFUpoD90000	geo:GSM8641038	[geo-gse282302]	2261	19144
2	Mc06lamGrOTYHhNF0000	geo:GSM8714181	[geo-gse274557]	419	36601
3	URKDREjVhAsRwwF10000	geo:GSM8714178	[geo-gse274557]	660	36601
4	StreGVL1Uyj2EeoM0000	geo:GSM8641015	[geo-gse282302]	2302	19144

Which key keeps an experiment together#

split(by=...) assigns whole groups to folds, and the default group is study_id. For GEO data the study_id is the sample accession, so every section is its own group, and sections of one series can land in different folds:

def series_per_fold(folds: dict[str, SpatialDataCohort]) -> dict[str, list[str]]:
    """The GEO series in each fold, from the collections column."""
    return {fold: sorted({c for cs in members.members["collections"] for c in cs}) for fold, members in folds.items()}


by_sample = cohort.split(by="study_id", train=0.6, val=0.2, test=0.2, seed=0)
series_per_fold(by_sample)

{'train': ['geo-gse274557', 'geo-gse294669'],
 'val': ['geo-gse282302'],
 'test': ['geo-gse282302']}

A series in two folds means that the test fold holds sections of an experiment the model was trained on. split(by=...) also takes a function that gets one member row and returns its group key. The function below returns the GEO series from the collections column, so each series stays in one fold:

def geo_series(row) -> str:
    """The GEO series of a member, from its collections."""
    return next(c for c in row["collections"] if c.startswith("geo-gse"))


folds = cohort.split(by=geo_series, train=0.4, val=0.3, test=0.3, seed=0)
series_per_fold(folds)

{'train': ['geo-gse294669'],
 'val': ['geo-gse274557'],
 'test': ['geo-gse282302']}

Fold sizes are counted in groups, not in sections, so with three series each fold gets one series, whatever the ratios. With more series the ratios apply as you would expect. For sources other than GEO, check which column groups an experiment (for example collections or doi) before you split.

A table adapter#

to_anndata(adapter=...) passes each member's table through an adapter before the join. An adapter is any object with the methods of the TableAdapter protocol, so it can check a member, drop it, or map its table into the space your model expects. The adapter below keeps the spots under the tissue and drops a section with fewer than 500 such spots.

class InTissueSpots:
    """Keep spots under the tissue; drop sections with fewer than ``min_spots`` of them."""

    name = "in-tissue-spots"
    version = "0.1.0"

    def __init__(self, min_spots: int = 200) -> None:
        self.min_spots = min_spots

    def requirements(self) -> dict[str, Any]:
        return {}  # read the whole table

    def validate_meta(self, row) -> list[str]:
        return [] if row["technology"] == "Visium" else [f"not Visium: {row['technology']}"]

    def validate(self, adata, row) -> list[str]:
        if "in_tissue" not in adata.obs:
            return ["no obs['in_tissue']"]
        n = int((adata.obs["in_tissue"] == 1).sum())
        return [] if n >= self.min_spots else [f"only {n} spots under the tissue"]

    def build(self, adata, row):
        return adata[adata.obs["in_tissue"] == 1].copy()


adapter = InTissueSpots(min_spots=500)
isinstance(adapter, TableAdapter)

True

One table per fold#

Each fold is a cohort, so to_anndata reads its tables and joins them. gene_key="symbol" puts the genes of every member under their symbols.

tables = {fold: members.to_anndata(gene_key="symbol", adapter=adapter) for fold, members in folds.items()}
for fold, adata in tables.items():
    print(f"{fold:5s} {adata.n_obs:5d} spots x {adata.n_vars} genes from {adata.obs['uid'].nunique()} sections")

train  1142 spots x 17943 genes from 1 sections
val     660 spots x 36601 genes from 1 sections
test   4563 spots x 19144 genes from 2 sections

Sections that the adapter dropped are listed in uns["sddb"]["dropped"] with the reason:

{fold: adata.uns.get("sddb", {}).get("dropped", {}) for fold, adata in tables.items()}

{'train': {},
 'val': {'Mc06lamGrOTYHhNF0000': 'only 419 spots under the tissue'},
 'test': {}}

The folds have different numbers of genes, because the series store different gene lists (17943, 36601 and 19144 genes). A model needs the same features in every fold, so keep the genes that all folds share, in one fixed order:

shared = sorted(set.intersection(*(set(adata.var_names) for adata in tables.values())))
tables = {fold: adata[:, shared].copy() for fold, adata in tables.items()}
print(len(shared), "genes in every fold")
{fold: adata.shape for fold, adata in tables.items()}

17943 genes in every fold

{'train': (1142, 17943), 'val': (660, 17943), 'test': (4563, 17943)}

X holds the counts and obs["uid"] the section of each spot, which is what a training loop needs.

train = tables["train"]
print(type(train.X).__name__, train.X.shape, f"{train.X.sum():.0f} counts")
train.obs["uid"].value_counts()

csr_matrix (1142, 17943) 197054 counts

uid
qEP9pd4LlyKeufe20000    1142
Name: count, dtype: int64

Record the split#

Every fold carries the same SplitManifest. It records the group column, the seed, the ratios, the fold of every uid and the hash of the cohort it was made from. Its by records the function name. Write it next to the model, together with the cohort manifest, which records the catalog release. Keep the grouping function in your code, because the manifest records only its name.

split = folds["train"].split_manifest
split.write(work / "split.json")
cohort.freeze().write(work / "cohort.json")

again = SplitManifest.load(work / "split.json")
print(again.by, again.seed, again.ratios)
print("same assignments:", again.assignments == split.assignments)
print("made from this cohort:", again.parent_hash == cohort.freeze().hash)

geo_series 0 {'test': 0.3, 'train': 0.4, 'val': 0.3}
same assignments: True
made from this cohort: True

split() is deterministic. The same cohort, column, ratios and seed give the same folds on any machine, because the groups are sorted before a seeded shuffle.

repeat = cohort.split(by=geo_series, train=0.4, val=0.3, test=0.3, seed=0)
repeat["train"].split_manifest.assignments == split.assignments

True