# Timestamps and tags

Two choices decide whether an extractor’s data lines up and stays separable: how its timestamps are read, and which tags it applies. The examples use the [Python package](https://dev.nominal.io/extractors/python.md); the same levels exist in the [manifest](https://dev.nominal.io/extractors/contract.md#multiple-output-files) and the [REST API](https://dev.nominal.io/extractors/register.md#rest-api).

## Timestamps

Nominal places every sample on an absolute timeline. Your output supplies a numeric or string timestamp per sample; timestamp metadata says how to read it.

### Where metadata comes from, and what wins

Three levels can specify it. Per output file, the pipeline resolves them in this order:

| Precedence | Level | Set with | Scope |
| --- | --- | --- | --- |
| 1 (highest) | Per output | `ctx.add_tabular(..., timestamp_column=, timestamp_type=)` | one file in one run |
| 2 | Ingest request | `dataset.add_containerized(..., timestamp_column=, timestamp_type=)` | every output of one run |
| 3 (lowest) | Image default | `@manifest_extractor(default_timestamp_column=, default_timestamp_type=)`, exported by `.registration_kwargs()` | every run of that image |

Levels 2 and 3 are resolved **before the container runs**, and ingestion fails if both are absent. That is why registration requires a default: it guarantees the job always has metadata to fall back to. Level 1 is applied afterward, from the manifest your code writes, and overrides the resolved job-level value for that file only.

Use each level for what it describes:

- **The image default** is the normal shape of this extractor’s output. Declare its column and type on the extractor decorator, then register using `.registration_kwargs()`. Single-file extractors accept the same timestamp options.
- **The ingest request** is for facts about *this upload* that the image cannot know. Above all, this is where a relative t0 belongs: `timestamp_type=ts.Relative("milliseconds", start=t0)`.
- **Per output** is for a run whose files disagree on timestamp fields or units. A manifest extractor writing telemetry in microseconds and events in seconds needs this; nothing else does.

Per-output metadata takes numeric types only, `ts.Epoch` or `ts.Relative`, in units of seconds through nanoseconds. An output needing ISO 8601 or a custom string format must omit the per-output pair and inherit the job-level value, which accepts the full range. For an Avro output, inheriting is correct only when the job-level type is numeric too: Avro timestamps are integers, and a string format cannot read them.

```python
# per output, in the manifest your extractor writes
ctx.add_tabular(part, timestamp_column="ts", timestamp_type="epoch_microseconds")
ctx.add_tabular(run, timestamp_column="elapsed", timestamp_type=ts.Relative("milliseconds", start=t0))

# avro takes the type alone: the schema fixes which field holds the timestamps
ctx.add_avro_stream(records, timestamp_type="epoch_microseconds")
```

### Prefer relative timestamps

Emit elapsed time from the start of the recording and supply t0 at ingest. Reach for absolute (`ts.Epoch`, `ts.Iso8601`, `ts.Custom`) only when you can guarantee every timestamp is already correct in absolute time.

Relative wins on two counts.

**It is recoverable.** With relative timestamps, absolute time is `(elapsed in the file) + (t0 in metadata)`, so a wrong t0 is a metadata error: delete the file and re-ingest the same bytes with a corrected `start`. With absolute timestamps, the times *are* the data, so the only fix is to download the file, rewrite every value, and upload it again.

**It is a smaller problem to get right.** Relative asks one question per sample, how far into the recording it is, which the logger measures directly off its own clock. Absolute asks every sample to be correct in wall-clock time, which depends on GPS lock, NTP sync, and clock drift, any of which can leave you with jumps, rounding, or an offset nobody notices until someone compares two sources.

So: if the source records only elapsed time, emit relative. If the source records absolute time but you cannot vouch for it (unsynced machines, intermittent GPS, timestamps rounded to the second, occasional jumps), convert to elapsed-from-first-sample and emit relative with a best-estimate t0. Choosing absolute is a claim that no timestamp will ever need altering.

Do **not** register `ts.Relative` as an image’s `default_timestamp_type`. A default is set once and applies to every future ingest, so a fixed `start` gives every file the same t0. Register an absolute default for the fallback and pass `Relative` per ingest, where each file carries its own start.

### Granularity

Use the unit the data has. Declaring `epoch_milliseconds` for microsecond data does not lose precision; it misplaces every sample by a factor of 1000.

## Tags

Tags separate otherwise-identical channels. A [channel](https://dev.nominal.io/platform/data-model.md#channels) is identified by name, so two stands both producing `chamber_pressure` blend into one series unless a tag distinguishes them. Blended data is harder to detect and harder to undo than a failed ingest.

Tags reach the data three ways, in increasing specificity:

- **The ingest request**: `add_containerized(..., tags={"vehicle": "n1234"})`, applied to everything the run produces and readable in the container as `ctx.additional_tags`. Use it for facts about *this upload* that the file does not carry.
- **A tag column**: `ctx.add_tabular(path, tag_columns={"motor": "motor_id"})`, read per row, when the distinguishing fact varies *within* a file.
- **Avro files** and **JSON Lines logs**, which carry tags within the files themselves.

A good tag identifies a *source*, is stable for the life of the data, and has few distinct values: vehicle, stand, motor serial, run identifier. A bad tag is a measurement (that is a channel), a value derived from a timestamp (that is the timeline), or free text that varies by upload: `Stand A`, `stand-a`, and `standA` become three unrelated series. Nothing normalizes tag values, so normalize them in the extractor rather than relying on each caller.

`channel_prefix` and tags solve different problems. A prefix renames channels (`engine/chamber_pressure`); a tag leaves the name alone and adds a dimension you can filter and group by. Prefer a tag when comparing the same measurement across sources.