# Python with Ibis

[Ibis](https://ibis-project.org/) builds SQL from Python dataframe expressions. The Nominal backend for Ibis sends that SQL to the [SQL interface](https://dev.nominal.io/sql-guides.md) and returns the results through Ibis’s output methods, such as `to_polars()` and `to_pandas()`. Queries through Ibis follow the same [rules](https://dev.nominal.io/sql-guides/writing-queries.md) and [limits](https://dev.nominal.io/sql-reference/rest-api/query.md#limits) as any other SQL query and read the tables described in the [Schema](https://dev.nominal.io/sql-reference/schema.md) reference. The [Ibis reference](https://dev.nominal.io/python/ref/ibis.md) documents the backend itself.

## Install and connect

With Python 3.11 or later, install the Nominal Python client with its `ibis` extra, which also installs Ibis and PyArrow:

```shell
pip install "nominal[ibis]"
```

Connect with the `default` profile from [API keys setup](https://dev.nominal.io/quickstart/api-keys-setup.md), which holds your API key, REST API URL, and workspace RID:

```python
import ibis
from nominal.core import NominalClient

client = NominalClient.from_profile("default")
con = ibis.nominal.connect(client)
print(con.list_tables())
```

## Find a dataset

[Every query on a telemetry table](https://dev.nominal.io/sql-guides/writing-queries.md#scoping-telemetry-queries) filters by `dataset_rid`. The `datasets` table lists the datasets you can read:

```python
datasets = con.table("datasets")
print(datasets.select("dataset_rid", "name").limit(20).to_polars())
```

Use a `dataset_rid` from that list, or [find a dataset RID](https://dev.nominal.io/quickstart/quick/quick-dataset-rid.md) in the Nominal app, for the examples below.

## Read a channel

Filter a telemetry table by dataset, channel, and time range, then select the columns you need. Compare `ts` with Python `datetime` values, which this example parses from ISO 8601 strings such as `2026-01-01T12:00:00Z`:

```python
from datetime import datetime

NOMINAL_DATASET_RID = "<ri.catalog.instance.dataset.identifier>"

points = con.table("points_double")
query = (
    points.filter(
        points.dataset_rid == NOMINAL_DATASET_RID,
        points.channel == "<channel name>",
        points.ts >= datetime.fromisoformat("<start time>"),
        points.ts < datetime.fromisoformat("<end time>"),
    )
    .select("ts", "value")
    .order_by("ts")
)
df = query.to_polars()
```

Swap `to_polars()` for another Ibis output method, such as `to_pandas()`, to get the result in another format, or write the result to a file with `to_parquet("<path>")`. Timestamps come back in UTC. In filters, a `datetime` without a time zone is read as UTC, and one with a time zone is converted to UTC.

To narrow the query to one source, as in [Filtering by tag](https://dev.nominal.io/sql-guides/writing-queries.md#filtering-by-tag), add `points.tags["<tag key>"] == "<tag value>"` to the filter.

`print(con.compile(query))` shows the SQL that Ibis sends. When the SQL interface rejects a query, Ibis raises a `NominalSqlError` that explains why, for example a missing `dataset_rid` filter.

## Downsample and aggregate

Group by a time bucket to turn samples into per-minute statistics:

```python
NOMINAL_DATASET_RID = "<ri.catalog.instance.dataset.identifier>"

points = con.table("points_double")
channel = points.filter(
    points.dataset_rid == NOMINAL_DATASET_RID,
    points.channel == "<channel name>",
)
per_minute = (
    channel.group_by(minute=channel.ts.bucket(minutes=1))
    .agg(
        mean=channel.value.mean(),
        min=channel.value.min(),
        max=channel.value.max(),
    )
    .order_by("minute")
    .to_polars()
)
```

`bucket()`, used above, and `truncate()`, such as `channel.ts.truncate("m")` for whole minutes, both round timestamps down to a time interval. Ibis returns the results of both without a time zone. To keep the results in UTC like `ts`, add `.cast("timestamp('UTC')")`, as in `channel.ts.bucket(minutes=1).cast("timestamp('UTC')")`.

To get each channel’s latest reading, group by channel and take `value.argmax(rows.ts)`, the `value` from the row with the highest `ts`:

```python
NOMINAL_DATASET_RID = "<ri.catalog.instance.dataset.identifier>"

points = con.table("points_double")
rows = points.filter(points.dataset_rid == NOMINAL_DATASET_RID)
latest = (
    rows.group_by("channel")
    .agg(last_value=rows.value.argmax(rows.ts), last_seen=rows.ts.max())
    .order_by("channel")
    .to_polars()
)
```

[Aggregate functions](https://dev.nominal.io/sql-reference/schema/function-reference.md#aggregate-functions) lists the other aggregates the SQL interface supports.

## Run SQL as written

`con.raw_sql()` sends a SQL string exactly as written and returns a [PyArrow table](https://arrow.apache.org/docs/python/generated/pyarrow.Table.html). Use `con.raw_sql()` for SQL you already have, such as a query from the SQL console or a `PIVOT` from [Combining telemetry tables](https://dev.nominal.io/sql-guides/writing-queries.md#combining-telemetry-tables). This example reads struct channel values from `points_struct`, which stores each sample as JSON text:

```python
NOMINAL_DATASET_RID = "<ri.catalog.instance.dataset.identifier>"

table = con.raw_sql(f"""
    SELECT ts, channel, value
    FROM points_struct
    WHERE dataset_rid = '{NOMINAL_DATASET_RID}'
    ORDER BY ts
    LIMIT 100
""")
```

## Read large results in batches

`to_pyarrow_batches()` streams a result as PyArrow record batches, so a large result never has to fit in memory at once. This example streams the `query` built in [Read a channel](https://dev.nominal.io/sql-guides/ibis.md#read-a-channel):

```python
with query.to_pyarrow_batches() as reader:
    for batch in reader:
        print(batch.num_rows)
```