# The container contract

This is the contract **every** extractor image must implement, regardless of language or base image. Nominal runs your container in isolation, with your input files mounted read-only and a writable output directory.

> **Tip**
>
> **Working in Python?** [Write an extractor in Python](https://dev.nominal.io/extractors/python.md) covers helpers that implement this contract for you: you declare inputs and outputs, and the package reads the environment and writes `manifest.json`. This page describes the contract itself, for images in any language.

## Environment

When your container starts, Nominal sets these environment variables:

- **One variable per declared input**, named by the environment variable you chose when [registering the image](https://dev.nominal.io/extractors/register.md). Each is set to the path of the mounted input file (for example `INPUT_FILE=/input/data.bin`). Input files are mounted under `/input`.
- **One variable per declared parameter**, named by its environment variable, set to the value supplied for that ingest. Parameter values are always strings.
- **`OUTPUT_DIR`**: the absolute path of the directory your container must write its output into.

Your container must:

1. Read each input file from the environment variable you declared for it.
2. Read each parameter value from its environment variable, if your extractor declares any.
3. Write output into `$OUTPUT_DIR`: either a [single file](https://dev.nominal.io/extractors/contract.md#single-output-file) in the declared output format, or [multiple files](https://dev.nominal.io/extractors/contract.md#multiple-output-files) described by a `manifest.json`.
4. Exit `0` on success. A non-zero exit code fails the ingest.

## Inputs and parameters

Parameters are tuning knobs. Inputs are the data to extract from.

- **Input**: a file the extraction reads, whether the capture itself or a sidecar it needs, such as a calibration table, a channel map, or a vendor schema.
- **Parameter**: a scalar that changes how the extraction runs, such as a threshold, a mode, or a sample-rate divisor.

A value that never varies between ingests belongs in the image rather than either mechanism. Structured configuration belongs in an input, not a parameter: parameter values are strings with no schema, so a mapping, a list, or a nested document has to be encoded and parsed by hand.

**To accept a variable number of files, or a directory layout, register a `.zip` or `.tar` input.** Each input is a single file, so archives are how you pass many files or preserve folder structure: declare the `zip` suffix for the input and unpack it in the extractor. Use this for loggers that emit a session directory rather than one capture, or for a run whose file count is not known in advance.

An input’s file suffixes also drive discovery: searching for extractors by file extension matches on them, so they determine which extractors a given file is offered for. The suffixes are descriptive rather than enforced: a `.txt` file sent to an input registered for `json` still uploads.

## Output formats

The output format you choose when registering the image fixes which of two output contracts the pipeline applies.

| Registered output format | Your container writes |
| --- | --- |
| `MANIFEST` | Any number of files, each described in `manifest.json` |
| `PARQUET`, `CSV`, `AVRO_STREAM` | Exactly one file, in that format |

**Write new extractors against the manifest contract.** It is a strict superset of the single-file one: one image can emit several files, mix telemetry with logs and video, and set per-file timestamps, tag columns, and channel prefixes. The single-file contract is the original one, kept for images already registered against it. Changing an image’s output format later requires registering a new image.

Those four are the only registerable formats. The other values on the output format enum cannot be ingested, so an image registered with one produces an extractor whose every ingest fails.

### Single output file

Read the input path from your declared environment variable, process it, and write one file to `$OUTPUT_DIR` in the output format you registered:

```python
import os
import pandas as pd

def main():
    # Read input file path from environment variable
    input_file = os.getenv("INPUT_FILE")
    if not input_file:
        raise ValueError("INPUT_FILE environment variable not set")

    # Process your data
    df = pd.read_csv(input_file)
    processed_df = df.dropna().reset_index(drop=True)

    # Write to output directory
    output_dir = os.getenv("OUTPUT_DIR")
    output_path = os.path.join(output_dir, "processed_data.parquet")
    processed_df.to_parquet(output_path, index=False)

if __name__ == "__main__":
    main()
```

### Multiple output files

To emit more than one output file from a single run:

- Write all output files (and any nested directories) into `$OUTPUT_DIR`.
- Write a `manifest.json` at the top level of `$OUTPUT_DIR` describing each output.

Each entry in the manifest’s `outputs` array must include:

- **`ingestType`**: one of `TABULAR` (CSV or Parquet), `AVRO_STREAM`, or `JSON_L` (JSON Lines logs).
- **`relativePath`**: the file’s path relative to `$OUTPUT_DIR`.
- **`tagColumns`** *(optional)*: a map of tag name to the column that supplies its value.
- **`channelPrefix`** *(optional)*: a prefix applied to every [channel](https://dev.nominal.io/platform/data-model.md#channels) from this file.
- **`timestampMetadata`** *(optional)*: timestamp metadata for this file only, overriding the ingest’s. It holds `seriesName` (the column or top-level JSON field with the timestamps), `epochTimeUnit` (`SECONDS`, `MILLISECONDS`, `MICROSECONDS`, or `NANOSECONDS`), and optionally `relativeOffset`, a start time that makes the values offsets from it. Only numeric timestamps can be expressed here. See [Timestamps](https://dev.nominal.io/extractors/timestamps-and-tags.md#extractor-timestamps) for how this combines with the other levels.

A file left in `$OUTPUT_DIR` that the manifest doesn’t name is not ingested.

#### Example `manifest.json`

```json
{
  "outputs": [
    {
      "ingestType": "TABULAR",
      "relativePath": "telemetry/shards/flight_data.parquet",
      "tagColumns": {"vehicle_id": "veh_id", "mission_id": "msn_id"},
      "channelPrefix": "telemetry/"
    },
    {
      "ingestType": "TABULAR",
      "relativePath": "events/2025/06/events.csv",
      "tagColumns": {"vehicle_id": "veh_id"}
    },
    {
      "ingestType": "AVRO_STREAM",
      "relativePath": "streams/sensor/stream.avro"
    },
    {
      "ingestType": "JSON_L",
      "relativePath": "logs/system/system_logs.jsonl",
      "channelPrefix": "system."
    }
  ]
}
```

#### Videos

Video files go in a separate `videoOutputs` array. Each entry has a `relativePath`, the dataset `channel` the video becomes, and a `timestampManifest` saying how its frames are placed in time: either the video’s own presentation timestamps anchored at an absolute start, or a JSON sidecar file holding one absolute nanosecond timestamp per frame.

Video outputs require a recent version of the Nominal platform. An older ingest pipeline ignores them, and rejects a manifest whose only outputs are videos.

## Runtime environment

Three runtime facts constrain the image, and none of them depend on how you build it:

- **The container runs as a fixed non-root user**, not whatever `USER` the image declares, with capabilities dropped. Nothing may require root at runtime.
- **`$HOME` is a writable empty mount** that shadows whatever the build left at that path. A dependency installed into the home directory, for example by `pip install --user` or into a `~/.venv`, is gone when the parser runs. It surfaces as `ModuleNotFoundError` for a package the image demonstrably contains. Install outside the home directory.
- **`$OUTPUT_DIR` is writable** by that user and needs no ownership or permission setup.

## Build requirements

Nominal runs extractor images on **`linux/amd64`**, so the architecture has to be set at build time, and `arm64` is the default on Apple Silicon. Build, check the architecture, then save the tarball you will upload:

```bash
docker build --platform linux/amd64 -t bespoke-file-parser:v1 .

docker image inspect bespoke-file-parser:v1 --format 'os={{.Os}} arch={{.Architecture}}'
# want: os=linux arch=amd64

docker save bespoke-file-parser:v1 -o bespoke-file-parser-v1.tar
```

Read the built image’s architecture rather than trusting the flag: an earlier native build can still own the tag, and a Dockerfile pinning `FROM --platform=...` overrides the command line.

> **Warning**
>
> Registration does not check the tarball’s architecture. An `arm64` build registers, reports `READY`, and activates successfully, then fails at ingest with an exec format error. Rebuild for `linux/amd64`, with `--no-cache` if the build already reported success.

Save a single-platform image. Nominal receives an archive rather than a registry it can negotiate a platform with, so do not assume it picks `amd64` out of a multi-platform archive.

## Logs

Write logs to `stdout` and `stderr`. Any logging library (or plain `print`) works. Nominal collects them into a workspace logs dataset on the **Data sources** page for each ingest, capped at 1 MiB per job, and that dataset is the first place to look when a run fails.