In-Memory Inputs

Every pipeline input can be handed to CosmoForge as an in-memory array instead of a file path. This is for callers that already hold their data — an upstream component pipeline, or an interactive session — and should not have to round-trip it through disk just to run an analysis.

The design is recorded in ADR-0017 (file-or-array loading seams), under docs/adr/.

Two adapters, one contract

Every input is loadable through exactly two adapters, converging on a single in-memory contract:

the file adapter — a path in InputParams, parsed by a low-level reader;
the injection adapter — the object handed straight to the high-level class.

An injected object always wins over the corresponding path. The configuration still supplies the scalars (nside, lmax, nsims, …); only the data moves.

The injected object is defined as exactly what the reader would have returned — the injection adapter adds no new accepted forms.

The kwarg vocabulary

Kwarg	Shadows	In-memory contract
`mask`	`maskfile`	`(npix,)` or `(npix, nfields)`, pixel-indexed per `params.ordering`
`noise_cov1`, `noise_cov2`	`covmatfile1/2`	reduced `(n_active, n_active)`
`maps1`, `maps2`	`inputmapfile1/2`	reduced `(n_active, n_sims)`
`cls_data`	`inputclfile`	`{label: C_ell}` dict of physical C_ell
`fiducial_cls`	`fiducialfile`	as `cls_data`
`beam`	`beam_file`	2-D, ≥3 rows (T/E/B window functions)

mask, cls_data, fiducial_cls and beam live on Core, so they are accepted by Fisher, Spectra and PICSLike alike. maps1/maps2 are a Spectra/PICSLike seam — Fisher never reads maps.

A disk-free run

from qube import Fisher, Spectra

fisher = Fisher(
    params,
    mask=mask,
    noise_cov1=noise_cov,
    cls_data=cls,
    fiducial_cls=cls,
    beam=beam,
)
fisher.run()

spectra = Spectra(params, fisher=fisher, maps1=maps)
spectra.run()
power_spectra = spectra.get_power_spectra(mode="deconvolved")

Combine this with opt-in persistence — leave every out* path unset — and the run reads nothing and writes nothing. The working directory is untouched from end to end.

One contract worth reading twice

Every seam takes exactly what its own reader would have returned — so an injected array is used verbatim, and any conversion the file path performs on the way in has already happened by the time the reader returns.

That matters in one place: cls_data/fiducial_cls must be physical C_ell. The input_convention normalisation (e.g. D_ell → C_ell) is applied only on the file path, never to an injected array. Convert before you inject.

The active-pixel index

noise_cov1/noise_cov2 and maps1/maps2 are reduced — restricted to the active (unmasked) pixels and ordered by the active-pixel index. QML works on active pixels, so this is the domain, not an implementation detail: a full-sky noise covariance is (npix * nfields)², which is 72 GiB for QU at nside 64 against 0.72 GiB reduced at fsky 0.1.

You do not need a Fisher (or any framework object) to learn the ordering. It is a pure function of the mask:

from cosmocore import active_pixel_index

index = active_pixel_index(mask)          # mask: (npix, nfields)

maps = maps_full.reshape(nfields * npix, nsims)[index]   # (n_active, nsims)
cov = cov_full[np.ix_(index, index)]                     # (n_active, n_active)

Each entry of index gives that reduced row’s position in the flattened full-pixel (nfields * npix,) vector. This is the same function the framework itself uses, so the published ordering is the ordering (ADR-0017).

You are free to build the reduced covariance any way you like — the one-liner above is a convenience, not a requirement. A caller that can produce it directly (from a noise model, block by block, or streamed) never has to materialise the full matrix at all.

cosmocore.active_pixels(mask) returns the per-component index arrays if you need them individually.

Worked example

src/cosmoforge.qube/notebooks/in_memory_inputs.ipynb drives both adapters over the same data — reading each input with its low-level reader, then handing the arrays in with every input path pointed at a nonexistent location — and asserts the two agree bit-for-bit (identical Fisher matrix and identical power spectra).

Run it with the jupyter dependency group:

uv sync --group jupyter
uv run --group jupyter jupyter lab src/cosmoforge.qube/notebooks/in_memory_inputs.ipynb