# HDF5

HDF5 files have no fixed layout, so you read the arrays you need with [h5py](https://docs.h5py.org/), shape them into columns and upload them as tables.
Reading in slices keeps files larger than memory out of RAM.

The examples also need `pip install h5py`.

## Connect the client

This guide assumes you have [installed](https://dev.nominal.io/python/guides/install.md) the client and setup your [API keys](https://dev.nominal.io/python/guides/api-keys-setup.md).

```python
from nominal.core import NominalClient

client = NominalClient.from_profile("default")
```

## Inspect the file

This example writes a small sample file with one group of three sensor arrays:

```python
import h5py
import numpy as np

with h5py.File("sample.h5", "w") as f:
    group = f.create_group("sensors/imu")
    group["time_s"] = np.arange(10_000) / 100  # 100 Hz
    group["accel_x"] = np.random.randn(10_000)
    group["accel_y"] = np.random.randn(10_000)
```

List the file’s groups and datasets to find the arrays to upload:

```python
with h5py.File("sample.h5") as f:
    f.visititems(lambda name, obj: print(name, getattr(obj, "shape", "")))
```

```text
sensors
sensors/imu
sensors/imu/accel_x (10000,)
sensors/imu/accel_y (10000,)
sensors/imu/time_s (10000,)
```

## Upload in batches

Read a slice of each array into a pandas DataFrame and upload it with [`upload_dataframe_to_dataset()`](https://dev.nominal.io/python/ref/generated/nominal.thirdparty.pandas.upload_dataframe_to_dataset.md#nominal.thirdparty.pandas.upload_dataframe_to_dataset "nominal.thirdparty.pandas.upload_dataframe_to_dataset").
Each batch adds to the same dataset.

```python
from datetime import datetime, timezone

import pandas as pd

from nominal import ts
from nominal.thirdparty.pandas import upload_dataframe_to_dataset

dataset = client.create_dataset("IMU from HDF5")
recording_start = datetime(2026, 9, 10, 14, 38, tzinfo=timezone.utc)
batch_size = 2_500

with h5py.File("sample.h5") as f:
    imu = f["sensors/imu"]
    for i in range(0, imu["time_s"].shape[0], batch_size):
        batch = pd.DataFrame({name: imu[name][i : i + batch_size] for name in ["time_s", "accel_x", "accel_y"]})
        upload_dataframe_to_dataset(
            dataset,
            batch,
            timestamp_column="time_s",
            timestamp_type=ts.Relative("seconds", start=recording_start),
        )
```

`upload_dataframe_to_dataset` waits for each batch to finish ingesting.
Multidimensional arrays need flattening into one column per element first, for example with `array[:, j]`.
For timestamp options, see [Timestamp formats](https://dev.nominal.io/python/guides/datasets/csv.md#timestamp-formats).