nbdemo.data API#
Sample datasets for Nominal’s open-source projects.
Datasets are grouped by domain: aero, med and
rf.
Import one from its domain; each has a path, and where it makes sense a native object and a DataFrame:
from nbdemo.data.med import ecg_mitbih_119
ecg_mitbih_119.path # a Path, to pass to your own loader
ecg_mitbih_119.native # a wfdb.Record
ecg_mitbih_119.df # a pandas.DataFrame
- catalog(domain: str | None = None) list[Dataset]#
Return every dataset, or the datasets of one domain.
- Parameters:
domain
A domain name such as
"rf";Nonefor all domains.- Returns:
The datasets, ordered by domain and name. Reads manifests only; nothing is downloaded.
- get(dataset_id: str) Dataset#
Return a dataset by its
"<domain>/<name>", for when the name is only known at run time.- Parameters:
dataset_id
For example
"med/ecg_mitbih_119".- Raises:
AttributeError – There is no such dataset.
- cache_dir() Path#
Return the directory that fetched files are saved in.
It is
$NBDEMO_CACHEif set, otherwisenbdemoin$XDG_CACHE_HOME(~/.cacheby default).- Returns:
The cache directory. It may not exist yet.
Domains#
Each domain module resolves dataset names when you import them, so there is no fixed list of attributes.
dir() of a domain lists its datasets; the catalog shows them all.
The aero datasets. Import one by name; see nbdemo.data.catalog() for the list.
The med datasets. Import one by name; see nbdemo.data.catalog() for the list.
The rf datasets. Import one by name; see nbdemo.data.catalog() for the list.
Dataset#
- class Dataset(domain: str, root: Path)#
A sample dataset: its files, and the forms they can be read in.
Get one by importing it from its domain, which reads only its manifest:
from nbdemo.data.med import ecg_mitbih_119 ecg_mitbih_119.path # a Path, to pass to your own loader ecg_mitbih_119.native # a wfdb.Record ecg_mitbih_119.df # a pandas.DataFrame
Nothing is downloaded or parsed until you use
path,paths,nativeordf. Files that do not ship in the package are downloaded on first use and kept incache_dir().- Parameters:
domain
The domain the dataset is in, such as
"med".root
The folder holding
dataset.tomland the files that ship in the package.
- classmethod from_dir(root: str | Path, domain: str = 'local') Dataset#
Read a dataset from any folder that holds a
dataset.toml.Use it to try a manifest before adding the dataset to the package.
- Parameters:
root
The dataset’s folder; its name is the dataset’s name.
domain
A label for the dataset’s domain.
- Raises:
ManifestError – The manifest is missing a field or has an invalid one.
- property forms: tuple[str, ...]#
The forms this dataset has.
pathis always one;nativeanddfare where they apply.
- property paths: dict[str, Path]#
Every file of the dataset, by name, downloading any that are missing.
The files are in one folder, so a loader can find companions such as a
.heabeside its.dat.
- property path: Path#
The path of the dataset’s main file, downloading the dataset’s files if needed.
The other files are beside it; see
paths.
- property native: Any#
The dataset as its format’s own Python object, such as a
wfdb.Record.- Raises:
FormUnavailable – The format has no such object here.
MissingExtraError – The library that reads the format is not installed.
- property df: Any#
The dataset as a
pandas.DataFrame.- Raises:
FormUnavailable – The data is not tabular, such as binary samples.
MissingExtraError – pandas, or the library that reads the format, is not installed.
Errors#
- exception SamplesError#
Base class for errors raised by nbdemo.
- exception ManifestError#
A
dataset.tomlis missing a field or has an invalid one.
- exception ChecksumError#
A downloaded file does not match the sha256 in its manifest.
A dataset has no
nativeordfform.Subclasses
AttributeError, sohasattr(dataset, "df")isFalsefor such a dataset.
- exception MissingExtraError#
A library that a loader needs is not installed.