# Ingest service

The Ingest Service handles the data ingestion into Nominal/data store.

OpenAPI file: [`IngestService.json`](https://dev.nominal.io/api/_downloads/4fb2722e2d8f00c4f4c6752fe1ed9272/IngestService.json)

## Endpoints

### Ingest

POST `/ingest/v1/ingest`

Triggers an ingest job, allowing either creating a new dataset or uploading to an existing one. This endpoint is meant to supersede all other ingestion endpoints as their functionality gets migrated to this one.

Body

`options` [`IngestOptions`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestoptions) required
:   ​

    **dataflash**

    Set `type` to `"dataflash"`.

    `dataflash` [`DataflashOpts`](https://dev.nominal.io/api/legacy/ingest-service.md#dataflashopts)
    :   ​

        Fields of `DataflashOpts`

        `source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
        :   ​

            **s3**

            Set `type` to `"s3"`.

            `s3` [`S3IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#s3ingestsource)
            :   ​

                Fields of `S3IngestSource`

                `path` string required
                :   ​

            **gcs**

            Set `type` to `"gcs"`.

            `gcs` [`GcsIngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#gcsingestsource)
            :   ​

                Fields of `GcsIngestSource`

                `path` string required
                :   ​

            **presignedFile**

            Set `type` to `"presignedFile"`.

            `presignedFile` [`PresignedFileIngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#presignedfileingestsource)
            :   ​

                Fields of `PresignedFileIngestSource`

                `url` string required
                :   ​

            **fileStore**

            Set `type` to `"fileStore"`.

            `fileStore` [`FileStoreIngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#filestoreingestsource)
            :   ​

                Fields of `FileStoreIngestSource`

                `fileRevisionRid` string (RID) required
                :   ​

        `target` [`DatasetIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#datasetingesttarget) required
        :   ​

            **new**

            Set `type` to `"new"`.

            `new` [`NewDatasetIngestDestination`](https://dev.nominal.io/api/legacy/ingest-service.md#newdatasetingestdestination)
            :   ​

                Fields of `NewDatasetIngestDestination`

                `datasetName` string
                :   ​

                `datasetDescription` string
                :   ​

                `properties` map of `PropertyName` to string
                :   ​

                `labels` array of string
                :   ​

                `channelConfig` [`ChannelConfig`](https://dev.nominal.io/api/legacy/ingest-service.md#channelconfig)
                :   ​

                    Fields of `ChannelConfig`

                    `prefixTreeDelimiter` string
                    :   If set, will construct a prefix tree for channels of the dataset using the given delimiter.

                `workspace` string (RID)
                :   The workspace in which to create the dataset. If not provided, the dataset will be created in the default workspace for the user’s organization, if the default workspace for the organization is configured.

                `markingRids` array of string (RID)
                :   The markings to apply to the created dataset. If not provided, the dataset will be visible to all users in the same workspace.

            **existing**

            Set `type` to `"existing"`.

            `existing` [`ExistingDatasetIngestDestination`](https://dev.nominal.io/api/legacy/ingest-service.md#existingdatasetingestdestination)
            :   ​

                Fields of `ExistingDatasetIngestDestination`

                `datasetRid` string (RID) required
                :   ​

                `expandRunBounds` array of string (RID)
                :   The ingest will keep track of the range of timestamps in the ingested data and update the included run’s time bounds. For runs with no end time (uninitialized), it will always set the start time to the file’s start time and set the end time to the file’s end time. For runs with existing end times, it will only expand the bounds (earliest start time, latest end time).

        `ignoreInvalidTimestamps` boolean
        :   If true, skips dataflash records whose timestamp cannot be represented (corrupted/out-of-range) instead of failing the entire ingest. If not provided, defaults to false.

        `additionalFileTags` map of `TagName` to string
        :   Specifies a tag set to apply to all data in the file.

    **mcapProtobufTimeseries**

    Set `type` to `"mcapProtobufTimeseries"`.

    `mcapProtobufTimeseries` [`McapProtobufTimeseriesOpts`](https://dev.nominal.io/api/legacy/ingest-service.md#mcapprotobuftimeseriesopts)
    :   ​

        Fields of `McapProtobufTimeseriesOpts`

        `source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
        :   ​

        `target` [`DatasetIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#datasetingesttarget) required
        :   ​

        `channelFilter` [`McapChannels`](https://dev.nominal.io/api/legacy/ingest-service.md#mcapchannels) required
        :   ​

            **all**

            Set `type` to `"all"`.

            `all` [`Empty`](https://dev.nominal.io/api/legacy/ingest-service.md#empty)
            :   ​

            **include**

            Set `type` to `"include"`.

            `include` array of [`McapChannelLocator`](https://dev.nominal.io/api/legacy/ingest-service.md#mcapchannellocator)
            :   ​

                **topic**

                Set `type` to `"topic"`.

                `topic` string
                :   ​

                **id**

                Set `type` to `"id"`.

                `id` integer
                :   ​

            **exclude**

            Set `type` to `"exclude"`.

            `exclude` array of [`McapChannelLocator`](https://dev.nominal.io/api/legacy/ingest-service.md#mcapchannellocator)
            :   ​

        `timestampType` [`McapTimestampType`](https://dev.nominal.io/api/legacy/ingest-service.md#mcaptimestamptype) required
        :   ​

            **logTime**

            Set `type` to `"logTime"`.

            `logTime` [`LogTime`](https://dev.nominal.io/api/legacy/ingest-service.md#logtime)
            :   ​

        `ignoreInvalidTopics` boolean
        :   If true, skips invalid MCAP topics during ingestion instead of failing the entire ingest. If not provided, defaults to false.

        `additionalFileTags` map of `TagName` to string
        :   Specifies a tag set to apply to all data in the file.

    **journalJson**

    Set `type` to `"journalJson"`.

    `journalJson` [`JournalJsonOpts`](https://dev.nominal.io/api/legacy/ingest-service.md#journaljsonopts)
    :   ​

        Fields of `JournalJsonOpts`

        `source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
        :   ​

        `target` [`DatasetIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#datasetingesttarget) required
        :   ​

        `channel` string
        :   If provided, ingests logs to the given channel. By default, log data will be ingested to a channel named ‘logs’.

        `timestampMetadata` [`JournalTimestampMetadata`](https://dev.nominal.io/api/legacy/ingest-service.md#journaltimestampmetadata)
        :   Optional timestamp field for the JSON lines. When provided, each line is timestamped from this field instead of the journald default `__REALTIME_TIMESTAMP`, allowing non-journald JSON logs to ingest. Each line must still contain a `MESSAGE` field. Only numeric epoch timestamps are supported. When absent, the journald default (`__REALTIME_TIMESTAMP`, microseconds) is used.

            Fields of `JournalTimestampMetadata`

            `fieldName` string required
            :   ​

            `epochOfTimeUnit` [`EpochTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#epochtimestamp) required
            :   ​

                Fields of `EpochTimestamp`

                `timeUnit` [`TimeUnit`](https://dev.nominal.io/api/legacy/ingest-service.md#timeunit) required
                :   ​

                    Values of `TimeUnit`

                    `DAYS`
                    :   ​

                    `HOURS`
                    :   ​

                    `MINUTES`
                    :   ​

                    `SECONDS`
                    :   ​

                    `MILLISECONDS`
                    :   ​

                    `MICROSECONDS`
                    :   ​

                    `NANOSECONDS`
                    :   ​

    **csv**

    Set `type` to `"csv"`.

    `csv` [`CsvOpts`](https://dev.nominal.io/api/legacy/ingest-service.md#csvopts)
    :   ​

        Fields of `CsvOpts`

        `source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
        :   ​

        `target` [`DatasetIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#datasetingesttarget) required
        :   ​

        `timestampMetadata` [`TimestampMetadata`](https://dev.nominal.io/api/legacy/ingest-service.md#timestampmetadata) required
        :   ​

            Fields of `TimestampMetadata`

            `seriesName` string required
            :   ​

            `timestampType` [`TimestampType`](https://dev.nominal.io/api/legacy/ingest-service.md#timestamptype) required
            :   ​

                **relative**

                Set `type` to `"relative"`.

                `relative` [`RelativeTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#relativetimestamp)
                :   ​

                    Fields of `RelativeTimestamp`

                    `timeUnit` [`TimeUnit`](https://dev.nominal.io/api/legacy/ingest-service.md#timeunit) required
                    :   ​

                    `offset` string (date-time)
                    :   Starting timestamp to use when indexing the file. This field is required when uploading a new file to an existing dataset.

                **absolute**

                Set `type` to `"absolute"`.

                `absolute` [`AbsoluteTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#absolutetimestamp)
                :   ​

                    **iso8601**

                    Set `type` to `"iso8601"`.

                    `iso8601` [`Iso8601Timestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#iso8601timestamp)
                    :   ​

                    **epochOfTimeUnit**

                    Set `type` to `"epochOfTimeUnit"`.

                    `epochOfTimeUnit` [`EpochTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#epochtimestamp)
                    :   ​

                    **customFormat**

                    Set `type` to `"customFormat"`.

                    `customFormat` [`CustomTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#customtimestamp)
                    :   ​

                        Fields of `CustomTimestamp`

                        `format` string required
                        :   The format string should be in the format of the `DateTimeFormatter` class in Java.

                        `defaultYear` integer
                        :   Default year is accepted as an optional field for cases like IRIG time format and will be overridden by year in time format.

                        `defaultDayOfYear` integer
                        :   Default day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.

        `channelPrefix` string required
        :   ​

        `channelNameOverrides` map of `ColumnName` to string
        :   ​

        `tagKeysFromColumns` array of string deprecated
        :   **Deprecated:** Deprecated in favor of tagColumns.

        `tagColumns` map of `TagName` to string
        :   A map of tag names to column names to derive the tag values from.

        `additionalFileTags` map of `TagName` to string
        :   Specifies a tag set to apply to all data in the file.

        `excludeColumns` array of string
        :   A set of column names to exclude from ingestion. These columns will not be ingested as channels. Useful for excluding columns that contain unsupported data types like multidimensional arrays.

    **parquet**

    Set `type` to `"parquet"`.

    `parquet` [`ParquetOpts`](https://dev.nominal.io/api/legacy/ingest-service.md#parquetopts)
    :   ​

        Fields of `ParquetOpts`

        `source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
        :   ​

        `target` [`DatasetIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#datasetingesttarget) required
        :   ​

        `timestampMetadata` [`TimestampMetadata`](https://dev.nominal.io/api/legacy/ingest-service.md#timestampmetadata) required
        :   ​

        `channelPrefix` string required
        :   ​

        `channelNameOverrides` map of `ColumnName` to string
        :   ​

        `tagKeysFromColumns` array of string deprecated
        :   **Deprecated:** Deprecated in favor of tagColumns.

        `tagColumns` map of `TagName` to string
        :   A map of tag names to column names to derive the tag values from.

        `additionalFileTags` map of `TagName` to string
        :   Specifies a tag set to apply to all data in the file.

        `isArchive` boolean
        :   If true, the file is an archive. Supported archive formats include .tar, .tar.gz, and .zip. Only files ending in .parquet within the archive will be ingested. If field not provided, defaults to false.

        `excludeColumns` array of string
        :   A set of column names to exclude from ingestion. These columns will not be ingested as channels. Useful for excluding columns that contain unsupported data types like multidimensional arrays.

    **video**

    Set `type` to `"video"`.

    `video` [`VideoOpts`](https://dev.nominal.io/api/legacy/ingest-service.md#videoopts) deprecated
    :   **Deprecated:** Legacy Video-entity ingest. Use videoV2, which ingests video as a channel on a dataset.

        Fields of `VideoOpts`

        `source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
        :   ​

        `target` [`VideoIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#videoingesttarget) required
        :   ​

            **new**

            Set `type` to `"new"`.

            `new` [`NewVideoIngestDestination`](https://dev.nominal.io/api/legacy/ingest-service.md#newvideoingestdestination)
            :   ​

                Fields of `NewVideoIngestDestination`

                `title` string
                :   Title of the Video that will get created. If not provided, a name is deduced from the ingested file.

                `description` string
                :   Description that is applied to the newly created video

                `properties` map of `PropertyName` to string
                :   Key-Value properties that are applied to the newly created video

                `labels` array of string
                :   Labels that are applied to the newly created video

                `videoFileDetails` [`VideoFileIngestDetails`](https://dev.nominal.io/api/legacy/ingest-service.md#videofileingestdetails)
                :   Metadata to associate with any created video file

                    Fields of `VideoFileIngestDetails`

                    `fileDescription` string
                    :   Description that is applied to the newly created video file. If not provided, defaults to the description used to create the video.

                    `fileProperties` map of `PropertyName` to string deprecated
                    :   Key-Value properties that are applied to the newly created video file. **Deprecated:** Field is ignored as video files do not have properties. Will be removed after 4/5/2025

                    `fileLabels` array of string deprecated
                    :   Labels that are applied to the newly created video file. **Deprecated:** Field is ignored as video files do not have labels. Will be removed after 4/5/2025

                `workspace` string (RID)
                :   The workspace in which to create the video. If not provided, the video will be created in the default workspace for the user’s organization, if the default workspace for the organization is configured.

                `markingRids` array of string (RID)
                :   The markings to apply to the created video. If not provided, the video will be visible to all users in the same workspace.

            **existing**

            Set `type` to `"existing"`.

            `existing` [`ExistingVideoIngestDestination`](https://dev.nominal.io/api/legacy/ingest-service.md#existingvideoingestdestination)
            :   ​

                Fields of `ExistingVideoIngestDestination`

                `videoRid` string (RID) required
                :   RID of the video to ingest the newly created video file to.

                `videoFileDetails` [`VideoFileIngestDetails`](https://dev.nominal.io/api/legacy/ingest-service.md#videofileingestdetails)
                :   Metadata to associate with any created video file

        `timestampManifest` [`VideoFileTimestampManifest`](https://dev.nominal.io/api/legacy/ingest-service.md#videofiletimestampmanifest) required
        :   ​

            **mcap**

            Set `type` to `"mcap"`.

            `mcap` [`McapTimestampManifest`](https://dev.nominal.io/api/legacy/ingest-service.md#mcaptimestampmanifest)
            :   ​

                Fields of `McapTimestampManifest`

                `mcapChannelLocator` [`McapChannelLocator`](https://dev.nominal.io/api/legacy/ingest-service.md#mcapchannellocator) required
                :   ​

            **s3path**

            Set `type` to `"s3path"`.

            `s3path` string
            :   ​

            **s3paths**

            Set `type` to `"s3paths"`.

            `s3paths` array of string
            :   Multiple sidecar manifest files whose per-frame timestamps are concatenated in list order. Used when a video’s frame timestamps are recorded across several files.

            **noManifest**

            Set `type` to `"noManifest"`.

            `noManifest` [`video.api.NoTimestampManifest`](https://dev.nominal.io/api/legacy/ingest-service.md#video-api-notimestampmanifest)
            :   ​

                Fields of `video.api.NoTimestampManifest`

                `startingTimestamp` [`Timestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#timestamp) required
                :   Specifies the original starting timestamp of the video.

                    Fields of `Timestamp`

                    `seconds` integer (int64) required
                    :   ​

                    `nanos` integer (int64) required
                    :   ​

                `scaleParameter` [`video.api.ScaleParameter`](https://dev.nominal.io/api/legacy/ingest-service.md#video-api-scaleparameter)
                :   A field that specifies that the frame rate of the video does not match the frame rate of the camera | i.e. a slowed down or sped up video. Can specify either the camera frame rate or the absolute end time.

                    **trueFrameRate**

                    Set `type` to `"trueFrameRate"`.

                    `trueFrameRate` number
                    :   ​

                    **endingTimestamp**

                    Set `type` to `"endingTimestamp"`.

                    `endingTimestamp` [`Timestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#timestamp)
                    :   ​

                    **scaleFactor**

                    Set `type` to `"scaleFactor"`.

                    `scaleFactor` number
                    :   the scale factor can be used to calculate whether media duration differs from a video’s | real duration, and if so, the true frame rate of the camera. The video time will thus be scaled | by the ratio of the real duration to media duration, or media frame rate to true frame rate.

        `overWriteSegments` boolean
        :   If true, overlapping segments from other video files within the same video will be deleted before inserting new segments. The cached segment metadata for affected files will be recomputed.

    **videoV2**

    Set `type` to `"videoV2"`.

    `videoV2` [`VideoOptsV2`](https://dev.nominal.io/api/legacy/ingest-service.md#videooptsv2)
    :   ​

        Fields of `VideoOptsV2`

        `source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
        :   ​

        `target` [`DatasetIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#datasetingesttarget) required
        :   ​

        `timestampManifest` [`VideoFileTimestampManifest`](https://dev.nominal.io/api/legacy/ingest-service.md#videofiletimestampmanifest) required
        :   ​

        `channel` string required
        :   ​

        `tags` map of `TagName` to string
        :   ​

        `overWriteSegments` boolean
        :   If true, overlapping segments from other dataset files within the same series will be deleted before inserting new segments. The cached segment metadata for affected files will be recomputed.

    **containerized**

    Set `type` to `"containerized"`.

    `containerized` [`ContainerizedOpts`](https://dev.nominal.io/api/legacy/ingest-service.md#containerizedopts)
    :   ​

        Fields of `ContainerizedOpts`

        `sources` map of `EnvironmentVariable` to [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource)
        :   ​

        `arguments` map of `EnvironmentVariable` to string
        :   ​

        `extractorRid` string (RID) required
        :   ​

        `timestampMetadata` [`TimestampMetadata`](https://dev.nominal.io/api/legacy/ingest-service.md#timestampmetadata)
        :   ​

        `tag` string
        :   ​

        `target` [`DatasetIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#datasetingesttarget) required
        :   ​

        `additionalFileTags` map of `TagName` to string
        :   Specifies a tag set to apply to all data in the file.

    **avroStream**

    Set `type` to `"avroStream"`.

    `avroStream` [`AvroStreamOpts`](https://dev.nominal.io/api/legacy/ingest-service.md#avrostreamopts)
    :   ​

        Fields of `AvroStreamOpts`

        `source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
        :   ​

        `target` [`DatasetIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#datasetingesttarget) required
        :   ​

        `additionalFileTags` map of `TagName` to string
        :   Specifies a tag set to apply to all data in the file.

        `timestampType` [`AvroNumericTimestampType`](https://dev.nominal.io/api/legacy/ingest-service.md#avronumerictimestamptype)
        :   How to interpret the numeric values in the `timestamps` array. Defaults to epoch-nanoseconds when omitted, matching the original contract of this API.

            **epoch**

            Set `type` to `"epoch"`.

            `epoch` [`EpochTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#epochtimestamp)
            :   ​

            **relative**

            Set `type` to `"relative"`.

            `relative` [`RelativeTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#relativetimestamp)
            :   ​

    **pointCloud**

    Set `type` to `"pointCloud"`.

    `pointCloud` [`PointCloudOpts`](https://dev.nominal.io/api/legacy/ingest-service.md#pointcloudopts)
    :   ​

        Fields of `PointCloudOpts`

        `source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
        :   ​

        `target` [`PointCloudIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#pointcloudingesttarget) required
        :   ​

            **existing**

            Set `type` to `"existing"`.

            `existing` [`ExistingSpatialIngestDestination`](https://dev.nominal.io/api/legacy/ingest-service.md#existingspatialingestdestination)
            :   ​

                Fields of `ExistingSpatialIngestDestination`

                `spatialRid` string (RID) required
                :   ​

        `daggerImportConfig` any JSON value required
        :   The Dagger **v2** `ImportRequest` body — everything from that schema except `source_uri`, which Nominal fills in from the resolved presigned URL for `source`. Forwarded to Dagger’s `POST /v2/imports/{model_uuid}`. Construct via the language’s generated dagger-client types (e.g. `dagger_client.models.ImportRequest` in Python). Carries the required `archetype` (attribute definitions and sampler configuration), an optional `format` (the source format and its format-specific parameters, including the `geometry_type` and `columns` a CSV source needs; omit it to let Dagger infer the format from the file extension), and an optional `quiche_config` — all of which are domain knowledge the client owns when it produced the file. The legacy v1 shape, with `geometry_type` and `columns` at the top level, is rejected.

        `sensorMetadata` [`PointCloudMetadata`](https://dev.nominal.io/api/legacy/ingest-service.md#pointcloudmetadata)
        :   Sensor metadata to associate with the point cloud.

            Fields of `PointCloudMetadata`

            `sensorModel` string
            :   The sensor model name, e.g. “Velodyne VLP-16”, “Ouster OS1-128”.

            `coordinateSystem` string
            :   Coordinate reference system identifier, e.g. “EPSG:4326”, “ENU”.

            `resolutionMm` number
            :   Spatial resolution of the point cloud in millimeters.

            `scanPattern` [`ScanPattern`](https://dev.nominal.io/api/legacy/ingest-service.md#scanpattern)
            :   The scan pattern of the lidar sensor.

                Values of `ScanPattern`

                `ROTATING`
                :   ​

                `SOLID_STATE`
                :   ​

                `FLASH`
                :   ​

                `MECHANICAL`
                :   ​

        `channel` string required
        :   Channel name for workbook integration.

        `tags` map of `TagName` to string
        :   Tags to apply to the point cloud series.

Response

`details` [`IngestDetails`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestdetails)
:   ​

    **dataset**

    Set `type` to `"dataset"`.

    `dataset` [`IngestDatasetFileDetails`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestdatasetfiledetails)
    :   ​

        Fields of `IngestDatasetFileDetails`

        `datasetFileId` string (UUID)
        :   ​

        `datasetRid` string (RID)
        :   ​

    **video**

    Set `type` to `"video"`.

    `video` [`IngestVideoFileDetails`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestvideofiledetails) deprecated
    :   **Deprecated:** Returned only for legacy Video-entity ingests; videoV2 ingests return dataset details carrying the created dataset file id.

        Fields of `IngestVideoFileDetails`

        `videoFileRid` string (RID)
        :   ​

        `videoRid` string (RID)
        :   ​

    **spatial**

    Set `type` to `"spatial"`.

    `spatial` [`IngestSpatialDetails`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestspatialdetails)
    :   ​

        Fields of `IngestSpatialDetails`

        `spatialRid` string (RID)
        :   ​

`ingestJobRid` string (RID)
:   ​

Request

**curl**

```shell
curl -X POST "https://api.gov.nominal.io/api/ingest/v1/ingest" \
  -H "Authorization: Bearer <api key>" \
  -H "Content-Type: application/json" \
  --data '{
  "options": {
    "type": "dataflash",
    "dataflash": {
      "source": {
        "type": "s3",
        "s3": {
          "path": "<path>"
        }
      },
      "target": {
        "type": "new",
        "new": {
          "datasetName": "<dataset name>",
          "datasetDescription": "<dataset description>",
          "properties": {
            "<key>": "<property value>"
          },
          "labels": [
            "<label>"
          ],
          "channelConfig": {
            "prefixTreeDelimiter": "<prefix tree delimiter>"
          },
          "workspace": "<workspace rid>",
          "markingRids": [
            "<marking rid>"
          ]
        }
      }
    }
  }
}'
```

**requests**

```python
import requests

response = requests.post(
    "https://api.gov.nominal.io/api/ingest/v1/ingest",
    headers={"Authorization": "Bearer <api key>"},
    json={
        "options": {
            "type": "dataflash",
            "dataflash": {
                "source": {"type": "s3", "s3": {"path": "<path>"}},
                "target": {
                    "type": "new",
                    "new": {
                        "datasetName": "<dataset name>",
                        "datasetDescription": "<dataset description>",
                        "properties": {"<key>": "<property value>"},
                        "labels": ["<label>"],
                        "channelConfig": {
                            "prefixTreeDelimiter": "<prefix tree delimiter>",
                        },
                        "workspace": "<workspace rid>",
                        "markingRids": ["<marking rid>"],
                    },
                },
            },
        },
    },
)
response.raise_for_status()
print(response.json())
```

**httpx**

```python
import httpx

response = httpx.post(
    "https://api.gov.nominal.io/api/ingest/v1/ingest",
    headers={"Authorization": "Bearer <api key>"},
    json={
        "options": {
            "type": "dataflash",
            "dataflash": {
                "source": {"type": "s3", "s3": {"path": "<path>"}},
                "target": {
                    "type": "new",
                    "new": {
                        "datasetName": "<dataset name>",
                        "datasetDescription": "<dataset description>",
                        "properties": {"<key>": "<property value>"},
                        "labels": ["<label>"],
                        "channelConfig": {
                            "prefixTreeDelimiter": "<prefix tree delimiter>",
                        },
                        "workspace": "<workspace rid>",
                        "markingRids": ["<marking rid>"],
                    },
                },
            },
        },
    },
)
response.raise_for_status()
print(response.json())
```

**aiohttp**

```python
import asyncio

import aiohttp

async def main() -> None:
    async with aiohttp.ClientSession() as session:
        async with session.post(
            "https://api.gov.nominal.io/api/ingest/v1/ingest",
            headers={"Authorization": "Bearer <api key>"},
            json={
                "options": {
                    "type": "dataflash",
                    "dataflash": {
                        "source": {"type": "s3", "s3": {"path": "<path>"}},
                        "target": {
                            "type": "new",
                            "new": {
                                "datasetName": "<dataset name>",
                                "datasetDescription": "<dataset description>",
                                "properties": {"<key>": "<property value>"},
                                "labels": ["<label>"],
                                "channelConfig": {
                                    "prefixTreeDelimiter": "<prefix tree delimiter>",
                                },
                                "workspace": "<workspace rid>",
                                "markingRids": ["<marking rid>"],
                            },
                        },
                    },
                },
            },
        ) as response:
            response.raise_for_status()
            print(await response.json())

asyncio.run(main())
```

Response: `200`

```json
{
  "details": {
    "type": "dataset",
    "dataset": {
      "datasetFileId": "<dataset file id>",
      "datasetRid": "<dataset rid>"
    }
  },
  "ingestJobRid": "<ingest job rid>"
}
```

### Rerun ingest

POST `/ingest/v1/re-ingest`

Triggers an ingest job using an existing ingest job RID. Returns the same response format as the /ingest endpoint.

Only a job that has finished can be re-run: re-running one still in flight races or duplicates the run already in progress. Cancel it first.

Body

`ingestJobRid` string (RID) required
:   ​

Response

`details` [`IngestDetails`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestdetails)
:   ​

    **dataset**

    Set `type` to `"dataset"`.

    `dataset` [`IngestDatasetFileDetails`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestdatasetfiledetails)
    :   ​

        Fields of `IngestDatasetFileDetails`

        `datasetFileId` string (UUID)
        :   ​

        `datasetRid` string (RID)
        :   ​

    **video**

    Set `type` to `"video"`.

    `video` [`IngestVideoFileDetails`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestvideofiledetails) deprecated
    :   **Deprecated:** Returned only for legacy Video-entity ingests; videoV2 ingests return dataset details carrying the created dataset file id.

        Fields of `IngestVideoFileDetails`

        `videoFileRid` string (RID)
        :   ​

        `videoRid` string (RID)
        :   ​

    **spatial**

    Set `type` to `"spatial"`.

    `spatial` [`IngestSpatialDetails`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestspatialdetails)
    :   ​

        Fields of `IngestSpatialDetails`

        `spatialRid` string (RID)
        :   ​

`ingestJobRid` string (RID)
:   ​

Request

**curl**

```shell
curl -X POST "https://api.gov.nominal.io/api/ingest/v1/re-ingest" \
  -H "Authorization: Bearer <api key>" \
  -H "Content-Type: application/json" \
  --data '{
  "ingestJobRid": "<ingest job rid>"
}'
```

**requests**

```python
import requests

response = requests.post(
    "https://api.gov.nominal.io/api/ingest/v1/re-ingest",
    headers={"Authorization": "Bearer <api key>"},
    json={"ingestJobRid": "<ingest job rid>"},
)
response.raise_for_status()
print(response.json())
```

**httpx**

```python
import httpx

response = httpx.post(
    "https://api.gov.nominal.io/api/ingest/v1/re-ingest",
    headers={"Authorization": "Bearer <api key>"},
    json={"ingestJobRid": "<ingest job rid>"},
)
response.raise_for_status()
print(response.json())
```

**aiohttp**

```python
import asyncio

import aiohttp

async def main() -> None:
    async with aiohttp.ClientSession() as session:
        async with session.post(
            "https://api.gov.nominal.io/api/ingest/v1/re-ingest",
            headers={"Authorization": "Bearer <api key>"},
            json={"ingestJobRid": "<ingest job rid>"},
        ) as response:
            response.raise_for_status()
            print(await response.json())

asyncio.run(main())
```

Response: `200`

```json
{
  "details": {
    "type": "dataset",
    "dataset": {
      "datasetFileId": "<dataset file id>",
      "datasetRid": "<dataset rid>"
    }
  },
  "ingestJobRid": "<ingest job rid>"
}
```

### Ingest run

POST `/ingest/v1/ingest-run`

Creates a run and ingests data sources to be added to the run.

Body

`rid` string (RID)
:   If a run with the same rid already exists, the run will be updated.

`title` string required
:   ​

`description` string required
:   ​

`startTime` [`UtcTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#utctimestamp) required
:   ​

    Fields of `UtcTimestamp`

    `secondsSinceEpoch` integer (int64) required
    :   ​

    `offsetNanoseconds` integer (int64)
    :   ​

`endTime` [`UtcTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#utctimestamp)
:   ​

`properties` map of `PropertyName` to string
:   ​

`labels` array of string
:   ​

`runPrefix` string
:   for example, SIM, HTL, FLT

`dataSources` map of `DataSourceRefName` to [`IngestRunDataSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestrundatasource)
:   ​

    Fields of `IngestRunDataSource`

    `dataSource` [`IngestDataSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestdatasource) required
    :   ​

        **existingDataSource**

        Set `type` to `"existingDataSource"`.

        `existingDataSource` string (RID)
        :   ​

        **newDataSource**

        Set `type` to `"newDataSource"`.

        `newDataSource` [`DeprecatedNewDataSource`](https://dev.nominal.io/api/legacy/ingest-service.md#deprecatednewdatasource)
        :   ​

            **csv**

            Set `type` to `"csv"`.

            `csv` [`DeprecatedNewCsv`](https://dev.nominal.io/api/legacy/ingest-service.md#deprecatednewcsv)
            :   ​

                Fields of `DeprecatedNewCsv`

                `source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
                :   ​

                    **s3**

                    Set `type` to `"s3"`.

                    `s3` [`S3IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#s3ingestsource)
                    :   ​

                        Fields of `S3IngestSource`

                        `path` string required
                        :   ​

                    **gcs**

                    Set `type` to `"gcs"`.

                    `gcs` [`GcsIngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#gcsingestsource)
                    :   ​

                        Fields of `GcsIngestSource`

                        `path` string required
                        :   ​

                    **presignedFile**

                    Set `type` to `"presignedFile"`.

                    `presignedFile` [`PresignedFileIngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#presignedfileingestsource)
                    :   ​

                        Fields of `PresignedFileIngestSource`

                        `url` string required
                        :   ​

                    **fileStore**

                    Set `type` to `"fileStore"`.

                    `fileStore` [`FileStoreIngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#filestoreingestsource)
                    :   ​

                        Fields of `FileStoreIngestSource`

                        `fileRevisionRid` string (RID) required
                        :   ​

                `name` string
                :   ​

                `properties` map of string to string
                :   ​

                `timeColumnSpec` [`TimestampMetadata`](https://dev.nominal.io/api/legacy/ingest-service.md#timestampmetadata)
                :   ​

                    Fields of `TimestampMetadata`

                    `seriesName` string required
                    :   ​

                    `timestampType` [`TimestampType`](https://dev.nominal.io/api/legacy/ingest-service.md#timestamptype) required
                    :   ​

                        **relative**

                        Set `type` to `"relative"`.

                        `relative` [`RelativeTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#relativetimestamp)
                        :   ​

                            Fields of `RelativeTimestamp`

                            `timeUnit` [`TimeUnit`](https://dev.nominal.io/api/legacy/ingest-service.md#timeunit) required
                            :   ​

                                Values of `TimeUnit`

                                `DAYS`
                                :   ​

                                `HOURS`
                                :   ​

                                `MINUTES`
                                :   ​

                                `SECONDS`
                                :   ​

                                `MILLISECONDS`
                                :   ​

                                `MICROSECONDS`
                                :   ​

                                `NANOSECONDS`
                                :   ​

                            `offset` string (date-time)
                            :   Starting timestamp to use when indexing the file. This field is required when uploading a new file to an existing dataset.

                        **absolute**

                        Set `type` to `"absolute"`.

                        `absolute` [`AbsoluteTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#absolutetimestamp)
                        :   ​

                            **iso8601**

                            Set `type` to `"iso8601"`.

                            `iso8601` [`Iso8601Timestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#iso8601timestamp)
                            :   ​

                            **epochOfTimeUnit**

                            Set `type` to `"epochOfTimeUnit"`.

                            `epochOfTimeUnit` [`EpochTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#epochtimestamp)
                            :   ​

                                Fields of `EpochTimestamp`

                                `timeUnit` [`TimeUnit`](https://dev.nominal.io/api/legacy/ingest-service.md#timeunit) required
                                :   ​

                            **customFormat**

                            Set `type` to `"customFormat"`.

                            `customFormat` [`CustomTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#customtimestamp)
                            :   ​

                                Fields of `CustomTimestamp`

                                `format` string required
                                :   The format string should be in the format of the `DateTimeFormatter` class in Java.

                                `defaultYear` integer
                                :   Default year is accepted as an optional field for cases like IRIG time format and will be overridden by year in time format.

                                `defaultDayOfYear` integer
                                :   Default day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.

                `channelConfig` [`ChannelConfig`](https://dev.nominal.io/api/legacy/ingest-service.md#channelconfig)
                :   ​

                    Fields of `ChannelConfig`

                    `prefixTreeDelimiter` string
                    :   If set, will construct a prefix tree for channels of the dataset using the given delimiter.

        **newDataSourceV2**

        Set `type` to `"newDataSourceV2"`.

        `newDataSourceV2` [`NewDataSource`](https://dev.nominal.io/api/legacy/ingest-service.md#newdatasource)
        :   ​

            Fields of `NewDataSource`

            `source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
            :   ​

            `properties` map of `PropertyName` to string
            :   ​

            `labels` array of string
            :   ​

            `description` string
            :   ​

            `name` string
            :   ​

            `timeColumnSpec` [`TimestampMetadata`](https://dev.nominal.io/api/legacy/ingest-service.md#timestampmetadata)
            :   ​

            `channelConfig` [`ChannelConfig`](https://dev.nominal.io/api/legacy/ingest-service.md#channelconfig)
            :   ​

    `timeOffsetSpec` [`TimeOffsetSpec`](https://dev.nominal.io/api/legacy/ingest-service.md#timeoffsetspec)
    :   ​

        **nanos**

        Set `type` to `"nanos"`.

        `nanos` [`Duration`](https://dev.nominal.io/api/legacy/ingest-service.md#duration)
        :   ​

            Fields of `Duration`

            `seconds` integer (int64) required
            :   ​

            `nanos` integer (int64) required
            :   ​

`workspace` string (RID)
:   The workspace in which to create the dataset. If not provided, the dataset will be created in the default workspace for the user’s organization, if the default workspace for the organization is configured.

Response

`runRid` string (RID)
:   ​

Request

**curl**

```shell
curl -X POST "https://api.gov.nominal.io/api/ingest/v1/ingest-run" \
  -H "Authorization: Bearer <api key>" \
  -H "Content-Type: application/json" \
  --data '{
  "title": "<title>",
  "description": "<description>",
  "startTime": {
    "secondsSinceEpoch": 1
  }
}'
```

**requests**

```python
import requests

response = requests.post(
    "https://api.gov.nominal.io/api/ingest/v1/ingest-run",
    headers={"Authorization": "Bearer <api key>"},
    json={
        "title": "<title>",
        "description": "<description>",
        "startTime": {"secondsSinceEpoch": 1},
    },
)
response.raise_for_status()
print(response.json())
```

**httpx**

```python
import httpx

response = httpx.post(
    "https://api.gov.nominal.io/api/ingest/v1/ingest-run",
    headers={"Authorization": "Bearer <api key>"},
    json={
        "title": "<title>",
        "description": "<description>",
        "startTime": {"secondsSinceEpoch": 1},
    },
)
response.raise_for_status()
print(response.json())
```

**aiohttp**

```python
import asyncio

import aiohttp

async def main() -> None:
    async with aiohttp.ClientSession() as session:
        async with session.post(
            "https://api.gov.nominal.io/api/ingest/v1/ingest-run",
            headers={"Authorization": "Bearer <api key>"},
            json={
                "title": "<title>",
                "description": "<description>",
                "startTime": {"secondsSinceEpoch": 1},
            },
        ) as response:
            response.raise_for_status()
            print(await response.json())

asyncio.run(main())
```

Response: `200`

```json
{
  "runRid": "<run rid>"
}
```

### Ingest video

POST `/ingest/v1/ingest-video`

**Deprecated:** Replaced by ingest.

Ingests video data from a S3 Nominal upload bucket.

Body

`sources` array of [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource)
:   ​

    **s3**

    Set `type` to `"s3"`.

    `s3` [`S3IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#s3ingestsource)
    :   ​

        Fields of `S3IngestSource`

        `path` string required
        :   ​

    **gcs**

    Set `type` to `"gcs"`.

    `gcs` [`GcsIngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#gcsingestsource)
    :   ​

        Fields of `GcsIngestSource`

        `path` string required
        :   ​

    **presignedFile**

    Set `type` to `"presignedFile"`.

    `presignedFile` [`PresignedFileIngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#presignedfileingestsource)
    :   ​

        Fields of `PresignedFileIngestSource`

        `url` string required
        :   ​

    **fileStore**

    Set `type` to `"fileStore"`.

    `fileStore` [`FileStoreIngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#filestoreingestsource)
    :   ​

        Fields of `FileStoreIngestSource`

        `fileRevisionRid` string (RID) required
        :   ​

`properties` map of `PropertyName` to string
:   ​

`labels` array of string
:   ​

`title` string
:   ​

`description` string
:   ​

`timestamps` [`VideoTimestampManifest`](https://dev.nominal.io/api/legacy/ingest-service.md#videotimestampmanifest) required
:   ​

    **noManifest**

    Set `type` to `"noManifest"`.

    `noManifest` [`ingest.api.NoTimestampManifest`](https://dev.nominal.io/api/legacy/ingest-service.md#ingest-api-notimestampmanifest)
    :   ​

        Fields of `ingest.api.NoTimestampManifest`

        `startingTimestamp` [`UtcTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#utctimestamp) required
        :   ​

            Fields of `UtcTimestamp`

            `secondsSinceEpoch` integer (int64) required
            :   ​

            `offsetNanoseconds` integer (int64)
            :   ​

        `scaleParameter` [`ingest.api.ScaleParameter`](https://dev.nominal.io/api/legacy/ingest-service.md#ingest-api-scaleparameter)
        :   A field that specifies that the frame rate of the video does not match the frame rate of the camera | i.e. a slowed down or sped up video. Can specify either the camera frame rate or the absolute end time.

            **trueFrameRate**

            Set `type` to `"trueFrameRate"`.

            `trueFrameRate` number
            :   ​

            **endingTimestamp**

            Set `type` to `"endingTimestamp"`.

            `endingTimestamp` [`UtcTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#utctimestamp)
            :   the timestamp corresponding to absolute starting timestamp plus absolute duration of the video.

            **scaleFactor**

            Set `type` to `"scaleFactor"`.

            `scaleFactor` number
            :   the scale factor can be used to calculate whether media duration differs from a video’s | real duration, and if so, the true frame rate of the camera. The video time will thus be scaled | by the ratio of the real duration to media duration, or media frame rate to true frame rate.

    **timestampManifests**

    Set `type` to `"timestampManifests"`.

    `timestampManifests` [`TimestampManifest`](https://dev.nominal.io/api/legacy/ingest-service.md#timestampmanifest)
    :   ​

        Fields of `TimestampManifest`

        `sources` array of [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource)
        :   ​

`workspace` string (RID)
:   The workspace in which to create the video. If not provided, the video will be created in the default workspace for the user’s organization, if the default workspace for the organization is configured.

Response

`videoRid` string (RID)
:   ​

`videoFileRid` string (RID)
:   ​

`asyncHandle` [`AsyncHandle`](https://dev.nominal.io/api/legacy/ingest-service.md#asynchandle) deprecated
:   **Deprecated:** Deprecated

    Fields of `AsyncHandle`

    `rid` string (RID)
    :   ​

Request

**curl**

```shell
curl -X POST "https://api.gov.nominal.io/api/ingest/v1/ingest-video" \
  -H "Authorization: Bearer <api key>" \
  -H "Content-Type: application/json" \
  --data '{
  "timestamps": {
    "type": "noManifest",
    "noManifest": {
      "startingTimestamp": {
        "secondsSinceEpoch": 1
      }
    }
  }
}'
```

**requests**

```python
import requests

response = requests.post(
    "https://api.gov.nominal.io/api/ingest/v1/ingest-video",
    headers={"Authorization": "Bearer <api key>"},
    json={
        "timestamps": {
            "type": "noManifest",
            "noManifest": {"startingTimestamp": {"secondsSinceEpoch": 1}},
        },
    },
)
response.raise_for_status()
print(response.json())
```

**httpx**

```python
import httpx

response = httpx.post(
    "https://api.gov.nominal.io/api/ingest/v1/ingest-video",
    headers={"Authorization": "Bearer <api key>"},
    json={
        "timestamps": {
            "type": "noManifest",
            "noManifest": {"startingTimestamp": {"secondsSinceEpoch": 1}},
        },
    },
)
response.raise_for_status()
print(response.json())
```

**aiohttp**

```python
import asyncio

import aiohttp

async def main() -> None:
    async with aiohttp.ClientSession() as session:
        async with session.post(
            "https://api.gov.nominal.io/api/ingest/v1/ingest-video",
            headers={"Authorization": "Bearer <api key>"},
            json={
                "timestamps": {
                    "type": "noManifest",
                    "noManifest": {"startingTimestamp": {"secondsSinceEpoch": 1}},
                },
            },
        ) as response:
            response.raise_for_status()
            print(await response.json())

asyncio.run(main())
```

Response: `200`

```json
{
  "videoRid": "<video rid>",
  "videoFileRid": "<video file rid>",
  "asyncHandle": {
    "rid": "<rid>"
  }
}
```

### Reingest from datasets

POST `/ingest/v1/reingest-dataset-files`

Re-ingests data from provided source datasets into either an existing target dataset, or a new one. Only supported for CSV and Parquet dataset files. Will only reingest dataset files and will drop streaming data from datasets.

Body

`sourceDatasets` array of string (RID)
:   The datasets to reingest data from. Can only include tagless, non-streaming datasets (V1). The datasets must be of the same granularity and must only include CSV or Parquet files. Will attempt to reingest from datasets in list order.

`targetDataset` [`DatasetIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#datasetingesttarget) required
:   The dataset to ingest data into. Can either be a new dataset or an existing dataset RID.

    **new**

    Set `type` to `"new"`.

    `new` [`NewDatasetIngestDestination`](https://dev.nominal.io/api/legacy/ingest-service.md#newdatasetingestdestination)
    :   ​

        Fields of `NewDatasetIngestDestination`

        `datasetName` string
        :   ​

        `datasetDescription` string
        :   ​

        `properties` map of `PropertyName` to string
        :   ​

        `labels` array of string
        :   ​

        `channelConfig` [`ChannelConfig`](https://dev.nominal.io/api/legacy/ingest-service.md#channelconfig)
        :   ​

            Fields of `ChannelConfig`

            `prefixTreeDelimiter` string
            :   If set, will construct a prefix tree for channels of the dataset using the given delimiter.

        `workspace` string (RID)
        :   The workspace in which to create the dataset. If not provided, the dataset will be created in the default workspace for the user’s organization, if the default workspace for the organization is configured.

        `markingRids` array of string (RID)
        :   The markings to apply to the created dataset. If not provided, the dataset will be visible to all users in the same workspace.

    **existing**

    Set `type` to `"existing"`.

    `existing` [`ExistingDatasetIngestDestination`](https://dev.nominal.io/api/legacy/ingest-service.md#existingdatasetingestdestination)
    :   ​

        Fields of `ExistingDatasetIngestDestination`

        `datasetRid` string (RID) required
        :   ​

        `expandRunBounds` array of string (RID)
        :   The ingest will keep track of the range of timestamps in the ingested data and update the included run’s time bounds. For runs with no end time (uninitialized), it will always set the start time to the file’s start time and set the end time to the file’s end time. For runs with existing end times, it will only expand the bounds (earliest start time, latest end time).

`ingestMetadata` map of `DatasetRid` to [`IngestMetadata`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestmetadata)
:   Mapping of dataset to ingest metadata. Ingest metadata only needs to be provided in request if it cannot be recovered from prior ingests. See documentation on each field to determine what metadata can be auto-recovered.

    Fields of `IngestMetadata`

    `timestampMetadata` [`TimestampMetadata`](https://dev.nominal.io/api/legacy/ingest-service.md#timestampmetadata)
    :   The timestamp metadata will be recovered from the dataset files if possible. Older datasets may have unrecoverable timestamp metadata. If unrecoverable, reingestion will throw MissingMetadataForReingest if not provided in request.

        Fields of `TimestampMetadata`

        `seriesName` string required
        :   ​

        `timestampType` [`TimestampType`](https://dev.nominal.io/api/legacy/ingest-service.md#timestamptype) required
        :   ​

            **relative**

            Set `type` to `"relative"`.

            `relative` [`RelativeTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#relativetimestamp)
            :   ​

                Fields of `RelativeTimestamp`

                `timeUnit` [`TimeUnit`](https://dev.nominal.io/api/legacy/ingest-service.md#timeunit) required
                :   ​

                    Values of `TimeUnit`

                    `DAYS`
                    :   ​

                    `HOURS`
                    :   ​

                    `MINUTES`
                    :   ​

                    `SECONDS`
                    :   ​

                    `MILLISECONDS`
                    :   ​

                    `MICROSECONDS`
                    :   ​

                    `NANOSECONDS`
                    :   ​

                `offset` string (date-time)
                :   Starting timestamp to use when indexing the file. This field is required when uploading a new file to an existing dataset.

            **absolute**

            Set `type` to `"absolute"`.

            `absolute` [`AbsoluteTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#absolutetimestamp)
            :   ​

                **iso8601**

                Set `type` to `"iso8601"`.

                `iso8601` [`Iso8601Timestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#iso8601timestamp)
                :   ​

                **epochOfTimeUnit**

                Set `type` to `"epochOfTimeUnit"`.

                `epochOfTimeUnit` [`EpochTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#epochtimestamp)
                :   ​

                    Fields of `EpochTimestamp`

                    `timeUnit` [`TimeUnit`](https://dev.nominal.io/api/legacy/ingest-service.md#timeunit) required
                    :   ​

                **customFormat**

                Set `type` to `"customFormat"`.

                `customFormat` [`CustomTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#customtimestamp)
                :   ​

                    Fields of `CustomTimestamp`

                    `format` string required
                    :   The format string should be in the format of the `DateTimeFormatter` class in Java.

                    `defaultYear` integer
                    :   Default year is accepted as an optional field for cases like IRIG time format and will be overridden by year in time format.

                    `defaultDayOfYear` integer
                    :   Default day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.

    `channelPrefix` string
    :   Channel prefix to use when reingesting the dataset. Defaults to empty string. Not recoverable from prior ingests and must be provided in request if needed.

    `channelNameOverrides` map of `ColumnName` to string
    :   Renames individual columns to explicit channel names. Composes with channelPrefix: `final name = channelPrefix + channelNameOverrides.getOrDefault(column, column)`. Defaults to empty map. Not recoverable from prior ingests and must be provided in request if needed. At most 1000 entries may be provided. Resolved channel names must be unique across columns; renaming a column onto another column’s name fails the ingest.

    `tagColumns` map of `TagName` to string
    :   A map of tag names to column names to derive the tag values from. Not recoverable from prior ingests and must be provided in request if needed.

    `additionalFileTags` map of `TagName` to string
    :   Additional tags to apply to all dataset files within the the given dataset. Not recoverable from prior ingests and must be provided in request if needed.

Response

`datasets` map of `DatasetRid` to [`ReingestFilesMapping`](https://dev.nominal.io/api/legacy/ingest-service.md#reingestfilesmapping)
:   ​

    Fields of `ReingestFilesMapping`

    `fileMap` map of `DatasetFileId` to [`IngestDetails`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestdetails)
    :   ​

        **dataset**

        Set `type` to `"dataset"`.

        `dataset` [`IngestDatasetFileDetails`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestdatasetfiledetails)
        :   ​

            Fields of `IngestDatasetFileDetails`

            `datasetFileId` string (UUID)
            :   ​

            `datasetRid` string (RID)
            :   ​

        **video**

        Set `type` to `"video"`.

        `video` [`IngestVideoFileDetails`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestvideofiledetails) deprecated
        :   **Deprecated:** Returned only for legacy Video-entity ingests; videoV2 ingests return dataset details carrying the created dataset file id.

            Fields of `IngestVideoFileDetails`

            `videoFileRid` string (RID)
            :   ​

            `videoRid` string (RID)
            :   ​

        **spatial**

        Set `type` to `"spatial"`.

        `spatial` [`IngestSpatialDetails`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestspatialdetails)
        :   ​

            Fields of `IngestSpatialDetails`

            `spatialRid` string (RID)
            :   ​

Request

**curl**

```shell
curl -X POST "https://api.gov.nominal.io/api/ingest/v1/reingest-dataset-files" \
  -H "Authorization: Bearer <api key>" \
  -H "Content-Type: application/json" \
  --data '{
  "targetDataset": {
    "type": "new",
    "new": {
      "datasetName": "<dataset name>",
      "datasetDescription": "<dataset description>",
      "properties": {
        "<key>": "<property value>"
      },
      "labels": [
        "<label>"
      ],
      "channelConfig": {
        "prefixTreeDelimiter": "<prefix tree delimiter>"
      },
      "workspace": "<workspace rid>",
      "markingRids": [
        "<marking rid>"
      ]
    }
  }
}'
```

**requests**

```python
import requests

response = requests.post(
    "https://api.gov.nominal.io/api/ingest/v1/reingest-dataset-files",
    headers={"Authorization": "Bearer <api key>"},
    json={
        "targetDataset": {
            "type": "new",
            "new": {
                "datasetName": "<dataset name>",
                "datasetDescription": "<dataset description>",
                "properties": {"<key>": "<property value>"},
                "labels": ["<label>"],
                "channelConfig": {"prefixTreeDelimiter": "<prefix tree delimiter>"},
                "workspace": "<workspace rid>",
                "markingRids": ["<marking rid>"],
            },
        },
    },
)
response.raise_for_status()
print(response.json())
```

**httpx**

```python
import httpx

response = httpx.post(
    "https://api.gov.nominal.io/api/ingest/v1/reingest-dataset-files",
    headers={"Authorization": "Bearer <api key>"},
    json={
        "targetDataset": {
            "type": "new",
            "new": {
                "datasetName": "<dataset name>",
                "datasetDescription": "<dataset description>",
                "properties": {"<key>": "<property value>"},
                "labels": ["<label>"],
                "channelConfig": {"prefixTreeDelimiter": "<prefix tree delimiter>"},
                "workspace": "<workspace rid>",
                "markingRids": ["<marking rid>"],
            },
        },
    },
)
response.raise_for_status()
print(response.json())
```

**aiohttp**

```python
import asyncio

import aiohttp

async def main() -> None:
    async with aiohttp.ClientSession() as session:
        async with session.post(
            "https://api.gov.nominal.io/api/ingest/v1/reingest-dataset-files",
            headers={"Authorization": "Bearer <api key>"},
            json={
                "targetDataset": {
                    "type": "new",
                    "new": {
                        "datasetName": "<dataset name>",
                        "datasetDescription": "<dataset description>",
                        "properties": {"<key>": "<property value>"},
                        "labels": ["<label>"],
                        "channelConfig": {
                            "prefixTreeDelimiter": "<prefix tree delimiter>",
                        },
                        "workspace": "<workspace rid>",
                        "markingRids": ["<marking rid>"],
                    },
                },
            },
        ) as response:
            response.raise_for_status()
            print(await response.json())

asyncio.run(main())
```

Response: `200`

```json
{
  "datasets": {
    "<key>": {
      "fileMap": {
        "<key>": {
          "type": "dataset",
          "dataset": {
            "datasetFileId": "<dataset file id>",
            "datasetRid": "<dataset rid>"
          }
        }
      }
    }
  }
}
```

### Delete file

DELETE `/ingest/v1/delete-file/{datasetRid}/file/{fileId}`

This is a best effort deletion of the file’s data based on the ingestedAt timestamp. This is an unreversible action. Only v2 dataset file deletion is supported. !!!WARNING!!! It’s possible that the file has overwritten points, in which case, those older points will not be recovered. Only use this endpoint if this is acceptable, the data across files are disjoint, or you’re willing to re-ingest files to manually recover older points.

Path parameters

`datasetRid` string (RID) required
:   ​

`fileId` string (UUID) required
:   ​

Response

No content (HTTP 204).

Request

**curl**

```shell
curl -X DELETE "https://api.gov.nominal.io/api/ingest/v1/delete-file/<dataset rid>/file/<file id>" \
  -H "Authorization: Bearer <api key>"
```

**requests**

```python
import requests

response = requests.delete(
    "https://api.gov.nominal.io/api/ingest/v1/delete-file/<dataset rid>/file/<file id>",
    headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
```

**httpx**

```python
import httpx

response = httpx.delete(
    "https://api.gov.nominal.io/api/ingest/v1/delete-file/<dataset rid>/file/<file id>",
    headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
```

**aiohttp**

```python
import asyncio

import aiohttp

async def main() -> None:
    async with aiohttp.ClientSession() as session:
        async with session.delete(
            "https://api.gov.nominal.io/api/ingest/v1/delete-file/<dataset rid>/file/<file id>",
            headers={"Authorization": "Bearer <api key>"},
        ) as response:
            response.raise_for_status()

asyncio.run(main())
```

Response: `204`

No content.

## Errors

[Errors](https://dev.nominal.io/api/legacy.md#errors) shows how these errors are sent.

Each error’s `errorName` is `UploadService:` followed by its name below.

| Status | Name | Notes |
| --- | --- | --- |
| 400 | `EmptyMultipartUpload` |  |
| 400 | `IncorrectSizeBytes` |  |
| 400 | `NullKey` |  |
| 400 | `SignDownloadPathOutsideUploadsBucket` |  |
| 400 | `UploadBucketNotAllowed` | A multipart-upload follow-up call specified a `bucket` this service does not recognize. It must be the value returned as `bucket` by initiateMultipartUpload, unchanged. |
| 400 | `WorkspaceResourceMismatch` | Both `workspace` and `resource` were set on the upload request, but the workspace derived from the resource does not match the explicitly-supplied workspace. Returned as a defensive sanity check so callers don’t accidentally believe they’re uploading to one workspace while the file is actually being routed to another. |
| 403 | `SignDownloadUnauthorized` |  |
| 404 | `ResourceNotFoundOrUnauthorized` | The resource referenced by an upload’s `resource` field was not found or the caller is not authorized to access it. Returned as NOT_FOUND (rather than PERMISSION_DENIED) to avoid leaking existence information about resources the caller cannot see. |
| 404 | `UploadObjectNotFound` | A multipart-upload follow-up call named a File Store object the ledger has no row for. A malformed key and an unknown one are reported identically, so a caller cannot tell which of the two it supplied. |

Each error’s `errorName` is `IngestService:` followed by its name below.

| Status | Name | Notes |
| --- | --- | --- |
| 400 | `BatchGranularityMismatch` | All dataset granularities must match. |
| 400 | `ContainerizedExtractorTimestampMetadataMissing` | Neither the containerized extractor (or its active container image) nor the ingest request provides timestamp metadata, so the extractor output cannot be indexed. Register the image with default timestamp metadata or supply it on the request. |
| 400 | `FailedToDownloadFileUserError` |  |
| 400 | `FileDeletionNotSupported` | The dataset file is part of a multi-file ingest job and cannot be deleted on its own. Delete the entire ingest job instead. |
| 400 | `FrameCountTimestampsMismatch` |  |
| 400 | `IncompatibleConnectionTypeAndLogQuery` |  |
| 400 | `IncompatibleFileForDataset` |  |
| 400 | `IncompatibleInputs` | The inputs provided to the ingest job are incompatible with each other |
| 400 | `IngestJobNotCompatibleForRerun` | The ingest job exists but is not compatible to be re-run. |
| 400 | `InvalidBucket` |  |
| 400 | `InvalidCustomTimestampFormat` |  |
| 400 | `InvalidS3Path` |  |
| 400 | `InvalidUrl` |  |
| 400 | `MissingMetadataForReingest` | The timestamp metadata required to reingest the dataset is missing and must be provided in the request. |
| 400 | `ParallelReingestIntoDatasetV1` | Cannot ingest multiple datasets in parallel into older datasets. It is recommended to reingest all source datasets and the target dataset into a new dataset destination. |
| 400 | `RecursiveReingest` | The target dataset is included in the source datasets. |
| 400 | `ReingestTooManyFiles` | The number of files to reingest is too large. |
| 400 | `RunBoundsInverted` |  |
| 400 | `UnsupportedIngestSource` |  |
| 400 | `UnsupportedRequestTypeForReIngest` |  |
| 403 | `ContainerizedExtractorUnauthorizedPasswordAccess` | The user is not authorized to access the containerized extractor password. |
| 404 | `DatasetFileNotFound` | The target dataset file could not be found. |
| 404 | `DatasetNotFound` | The target dataset could not be found. |
| 404 | `IngestJobNotFound` |  |
| 404 | `IngestTransformNotFound` | The ingest transform does not exist, or its parent ingest job is not visible to the caller. Returned as NOT_FOUND (rather than PERMISSION_DENIED) to avoid leaking existence information about transforms the caller cannot see. |
| 404 | `S3PathNotFound` |  |
| 404 | `StreamingSessionNotFound` | The streaming session could not be found. |
| 409 | `IngestJobNotCancellable` | The ingest job exists but is in a terminal state (COMPLETED, FAILED, or CANCELLED) and therefore cannot be cancelled. |
| 500 | `ContainerizedExtractorContainerFailed` |  |
| 500 | `FailedToDownloadFile` |  |
| 500 | `IngestJobMissingDatasetForIdempotencyKey` | A prior ingest job already exists for this idempotency key but never produced a dataset (the original submit did not get far enough). The idempotency key deduplicates to that existing job and does not re-run ingestion; rerun the job identified by ingestJobRid to complete it. |

## Types

### AbsoluteTimestamp

Set `type` to the name of one of the other fields, and set only that field.

`iso8601` [`Iso8601Timestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#iso8601timestamp)
:   ​

`epochOfTimeUnit` [`EpochTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#epochtimestamp)
:   ​

`customFormat` [`CustomTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#customtimestamp)
:   ​

### AsyncHandle

`rid` string (RID) required
:   ​

### AvroNumericTimestampType

Describes how the numeric `timestamps` column in an Avro Stream file should be interpreted. The Avro schema models `timestamps` as an array of longs, so only numeric timestamp variants are supported (no ISO-8601 / custom string formats).

Set `type` to the name of one of the other fields, and set only that field.

`epoch` [`EpochTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#epochtimestamp)
:   ​

`relative` [`RelativeTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#relativetimestamp)
:   ​

### AvroStreamOpts

Options for ingesting Avro data with the following schema. This is a “stream-like” file format to support use cases where a columnar/tabular format does not make sense. This closely matches Nominal’s streaming API, making it useful for use cases where network connection drops during streaming and a backup file needs to be created.

If this schema is not used, will result in a failed ingestion. { “type”: “record”, “name”: “AvroStream”, “namespace”: “io.nominal.ingest”, “fields”: [ { “name”: “channel”, “type”: “string”, “doc”: “Channel/series name (e.g., ‘vehicle_id’, ‘col_1’, ‘temperature’)”, }, { “name”: “timestamps”, “type”: {“type”: “array”, “items”: “long”}, “doc”: “Array of numeric timestamps; see timestampType for interpretation”, }, { “name”: “values”, “type”: {“type”: “array”, “items”: [“double”, “string”]}, “doc”: “Array of values. Can either be doubles or strings”, }, { “name”: “tags”, “type”: {“type”: “map”, “values”: “string”}, “default”: {}, “doc”: “Key-value metadata tags”, }, ], }

`source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
:   ​

`target` [`DatasetIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#datasetingesttarget) required
:   ​

`additionalFileTags` map of `TagName` to string
:   Specifies a tag set to apply to all data in the file.

`timestampType` [`AvroNumericTimestampType`](https://dev.nominal.io/api/legacy/ingest-service.md#avronumerictimestamptype)
:   How to interpret the numeric values in the `timestamps` array. Defaults to epoch-nanoseconds when omitted, matching the original contract of this API.

### ChannelConfig

`prefixTreeDelimiter` string
:   If set, will construct a prefix tree for channels of the dataset using the given delimiter.

### ContainerizedOpts

`sources` map of `EnvironmentVariable` to [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource)
:   ​

`arguments` map of `EnvironmentVariable` to string
:   ​

`extractorRid` string (RID) required
:   ​

`timestampMetadata` [`TimestampMetadata`](https://dev.nominal.io/api/legacy/ingest-service.md#timestampmetadata)
:   ​

`tag` string
:   ​

`target` [`DatasetIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#datasetingesttarget) required
:   ​

`additionalFileTags` map of `TagName` to string
:   Specifies a tag set to apply to all data in the file.

### CsvOpts

Options for ingesting csv files. Supported file formats include .csv, .csv.gz

`source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
:   ​

`target` [`DatasetIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#datasetingesttarget) required
:   ​

`timestampMetadata` [`TimestampMetadata`](https://dev.nominal.io/api/legacy/ingest-service.md#timestampmetadata) required
:   ​

`channelPrefix` string required
:   ​

`channelNameOverrides` map of `ColumnName` to string
:   ​

`tagKeysFromColumns` array of string deprecated
:   **Deprecated:** Deprecated in favor of tagColumns.

`tagColumns` map of `TagName` to string
:   A map of tag names to column names to derive the tag values from.

`additionalFileTags` map of `TagName` to string
:   Specifies a tag set to apply to all data in the file.

`excludeColumns` array of string
:   A set of column names to exclude from ingestion. These columns will not be ingested as channels. Useful for excluding columns that contain unsupported data types like multidimensional arrays.

### CustomTimestamp

`format` string required
:   The format string should be in the format of the `DateTimeFormatter` class in Java.

`defaultYear` integer
:   Default year is accepted as an optional field for cases like IRIG time format and will be overridden by year in time format.

`defaultDayOfYear` integer
:   Default day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.

### DataflashOpts

`source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
:   ​

`target` [`DatasetIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#datasetingesttarget) required
:   ​

`ignoreInvalidTimestamps` boolean
:   If true, skips dataflash records whose timestamp cannot be represented (corrupted/out-of-range) instead of failing the entire ingest. If not provided, defaults to false.

`additionalFileTags` map of `TagName` to string
:   Specifies a tag set to apply to all data in the file.

### DatasetIngestTarget

Set `type` to the name of one of the other fields, and set only that field.

`new` [`NewDatasetIngestDestination`](https://dev.nominal.io/api/legacy/ingest-service.md#newdatasetingestdestination)
:   ​

`existing` [`ExistingDatasetIngestDestination`](https://dev.nominal.io/api/legacy/ingest-service.md#existingdatasetingestdestination)
:   ​

### DeprecatedNewCsv

`source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
:   ​

`name` string
:   ​

`properties` map of string to string
:   ​

`timeColumnSpec` [`TimestampMetadata`](https://dev.nominal.io/api/legacy/ingest-service.md#timestampmetadata)
:   ​

`channelConfig` [`ChannelConfig`](https://dev.nominal.io/api/legacy/ingest-service.md#channelconfig)
:   ​

### DeprecatedNewDataSource

Set `type` to the name of one of the other fields, and set only that field.

`csv` [`DeprecatedNewCsv`](https://dev.nominal.io/api/legacy/ingest-service.md#deprecatednewcsv)
:   ​

### Duration

`seconds` integer (int64) required
:   ​

`nanos` integer (int64) required
:   ​

### Empty

An empty JSON object, `{}`.

### EpochTimestamp

`timeUnit` [`TimeUnit`](https://dev.nominal.io/api/legacy/ingest-service.md#timeunit) required
:   ​

### ExistingDatasetIngestDestination

`datasetRid` string (RID) required
:   ​

`expandRunBounds` array of string (RID)
:   The ingest will keep track of the range of timestamps in the ingested data and update the included run’s time bounds. For runs with no end time (uninitialized), it will always set the start time to the file’s start time and set the end time to the file’s end time. For runs with existing end times, it will only expand the bounds (earliest start time, latest end time).

### ExistingSpatialIngestDestination

`spatialRid` string (RID) required
:   ​

### ExistingVideoIngestDestination

`videoRid` string (RID) required
:   RID of the video to ingest the newly created video file to.

`videoFileDetails` [`VideoFileIngestDetails`](https://dev.nominal.io/api/legacy/ingest-service.md#videofileingestdetails)
:   Metadata to associate with any created video file

### FileStoreIngestSource

A resolved File Store revision to ingest, whose bytes are read in place from File Store storage.

`fileRevisionRid` string (RID) required
:   ​

### GcsIngestSource

`path` string required
:   ​

### ingest.api.NoTimestampManifest

`startingTimestamp` [`UtcTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#utctimestamp) required
:   ​

`scaleParameter` [`ingest.api.ScaleParameter`](https://dev.nominal.io/api/legacy/ingest-service.md#ingest-api-scaleparameter)
:   A field that specifies that the frame rate of the video does not match the frame rate of the camera | i.e. a slowed down or sped up video. Can specify either the camera frame rate or the absolute end time.

### ingest.api.ScaleParameter

Set `type` to the name of one of the other fields, and set only that field.

`trueFrameRate` number
:   ​

`endingTimestamp` [`UtcTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#utctimestamp)
:   the timestamp corresponding to absolute starting timestamp plus absolute duration of the video.

`scaleFactor` number
:   the scale factor can be used to calculate whether media duration differs from a video’s | real duration, and if so, the true frame rate of the camera. The video time will thus be scaled | by the ratio of the real duration to media duration, or media frame rate to true frame rate.

### IngestDatasetFileDetails

`datasetFileId` string (UUID)
:   ​

`datasetRid` string (RID) required
:   ​

### IngestDataSource

Set `type` to the name of one of the other fields, and set only that field.

`existingDataSource` string (RID)
:   ​

`newDataSource` [`DeprecatedNewDataSource`](https://dev.nominal.io/api/legacy/ingest-service.md#deprecatednewdatasource)
:   ​

`newDataSourceV2` [`NewDataSource`](https://dev.nominal.io/api/legacy/ingest-service.md#newdatasource)
:   ​

### IngestDetails

Set `type` to the name of one of the other fields, and set only that field.

`dataset` [`IngestDatasetFileDetails`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestdatasetfiledetails)
:   ​

`video` [`IngestVideoFileDetails`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestvideofiledetails) deprecated
:   **Deprecated:** Returned only for legacy Video-entity ingests; videoV2 ingests return dataset details carrying the created dataset file id.

`spatial` [`IngestSpatialDetails`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestspatialdetails)
:   ​

### IngestMetadata

`timestampMetadata` [`TimestampMetadata`](https://dev.nominal.io/api/legacy/ingest-service.md#timestampmetadata)
:   The timestamp metadata will be recovered from the dataset files if possible. Older datasets may have unrecoverable timestamp metadata. If unrecoverable, reingestion will throw MissingMetadataForReingest if not provided in request.

`channelPrefix` string
:   Channel prefix to use when reingesting the dataset. Defaults to empty string. Not recoverable from prior ingests and must be provided in request if needed.

`channelNameOverrides` map of `ColumnName` to string
:   Renames individual columns to explicit channel names. Composes with channelPrefix: `final name = channelPrefix + channelNameOverrides.getOrDefault(column, column)`. Defaults to empty map. Not recoverable from prior ingests and must be provided in request if needed. At most 1000 entries may be provided. Resolved channel names must be unique across columns; renaming a column onto another column’s name fails the ingest.

`tagColumns` map of `TagName` to string
:   A map of tag names to column names to derive the tag values from. Not recoverable from prior ingests and must be provided in request if needed.

`additionalFileTags` map of `TagName` to string
:   Additional tags to apply to all dataset files within the the given dataset. Not recoverable from prior ingests and must be provided in request if needed.

### IngestOptions

Set `type` to the name of one of the other fields, and set only that field.

`dataflash` [`DataflashOpts`](https://dev.nominal.io/api/legacy/ingest-service.md#dataflashopts)
:   ​

`mcapProtobufTimeseries` [`McapProtobufTimeseriesOpts`](https://dev.nominal.io/api/legacy/ingest-service.md#mcapprotobuftimeseriesopts)
:   ​

`journalJson` [`JournalJsonOpts`](https://dev.nominal.io/api/legacy/ingest-service.md#journaljsonopts)
:   ​

`csv` [`CsvOpts`](https://dev.nominal.io/api/legacy/ingest-service.md#csvopts)
:   ​

`parquet` [`ParquetOpts`](https://dev.nominal.io/api/legacy/ingest-service.md#parquetopts)
:   ​

`video` [`VideoOpts`](https://dev.nominal.io/api/legacy/ingest-service.md#videoopts) deprecated
:   **Deprecated:** Legacy Video-entity ingest. Use videoV2, which ingests video as a channel on a dataset.

`videoV2` [`VideoOptsV2`](https://dev.nominal.io/api/legacy/ingest-service.md#videooptsv2)
:   ​

`containerized` [`ContainerizedOpts`](https://dev.nominal.io/api/legacy/ingest-service.md#containerizedopts)
:   ​

`avroStream` [`AvroStreamOpts`](https://dev.nominal.io/api/legacy/ingest-service.md#avrostreamopts)
:   ​

`pointCloud` [`PointCloudOpts`](https://dev.nominal.io/api/legacy/ingest-service.md#pointcloudopts)
:   ​

### IngestRunDataSource

`dataSource` [`IngestDataSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestdatasource) required
:   ​

`timeOffsetSpec` [`TimeOffsetSpec`](https://dev.nominal.io/api/legacy/ingest-service.md#timeoffsetspec)
:   ​

### IngestSource

Set `type` to the name of one of the other fields, and set only that field.

`s3` [`S3IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#s3ingestsource)
:   ​

`gcs` [`GcsIngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#gcsingestsource)
:   ​

`presignedFile` [`PresignedFileIngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#presignedfileingestsource)
:   ​

`fileStore` [`FileStoreIngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#filestoreingestsource)
:   ​

### IngestSpatialDetails

`spatialRid` string (RID) required
:   ​

### IngestVideoFileDetails

`videoFileRid` string (RID) required
:   ​

`videoRid` string (RID) required
:   ​

### Iso8601Timestamp

An empty JSON object, `{}`.

### JournalJsonOpts

`source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
:   ​

`target` [`DatasetIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#datasetingesttarget) required
:   ​

`channel` string
:   If provided, ingests logs to the given channel. By default, log data will be ingested to a channel named ‘logs’.

`timestampMetadata` [`JournalTimestampMetadata`](https://dev.nominal.io/api/legacy/ingest-service.md#journaltimestampmetadata)
:   Optional timestamp field for the JSON lines. When provided, each line is timestamped from this field instead of the journald default `__REALTIME_TIMESTAMP`, allowing non-journald JSON logs to ingest. Each line must still contain a `MESSAGE` field. Only numeric epoch timestamps are supported. When absent, the journald default (`__REALTIME_TIMESTAMP`, microseconds) is used.

### JournalTimestampMetadata

`fieldName` string required
:   ​

`epochOfTimeUnit` [`EpochTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#epochtimestamp) required
:   ​

### LogTime

An empty JSON object, `{}`.

### McapChannelLocator

Locator for a channel in an mcap file. Channel name is not guaranteed to be unique, so channel ID should be used for mcap files with duplicate channel names.

Set `type` to the name of one of the other fields, and set only that field.

`topic` string
:   ​

`id` integer
:   ​

### McapChannels

Set `type` to the name of one of the other fields, and set only that field.

`all` [`Empty`](https://dev.nominal.io/api/legacy/ingest-service.md#empty)
:   ​

`include` array of [`McapChannelLocator`](https://dev.nominal.io/api/legacy/ingest-service.md#mcapchannellocator)
:   ​

`exclude` array of [`McapChannelLocator`](https://dev.nominal.io/api/legacy/ingest-service.md#mcapchannellocator)
:   ​

### McapProtobufTimeseriesOpts

`source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
:   ​

`target` [`DatasetIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#datasetingesttarget) required
:   ​

`channelFilter` [`McapChannels`](https://dev.nominal.io/api/legacy/ingest-service.md#mcapchannels) required
:   ​

`timestampType` [`McapTimestampType`](https://dev.nominal.io/api/legacy/ingest-service.md#mcaptimestamptype) required
:   ​

`ignoreInvalidTopics` boolean
:   If true, skips invalid MCAP topics during ingestion instead of failing the entire ingest. If not provided, defaults to false.

`additionalFileTags` map of `TagName` to string
:   Specifies a tag set to apply to all data in the file.

### McapTimestampManifest

Timestamps are derived from the mcap file containing the video frames.

`mcapChannelLocator` [`McapChannelLocator`](https://dev.nominal.io/api/legacy/ingest-service.md#mcapchannellocator) required
:   ​

### McapTimestampType

LogTime is default timestamp for MCAP messages and should be used in most cases.

Set `type` to the name of one of the other fields, and set only that field.

`logTime` [`LogTime`](https://dev.nominal.io/api/legacy/ingest-service.md#logtime)
:   ​

### NewDatasetIngestDestination

`datasetName` string
:   ​

`datasetDescription` string
:   ​

`properties` map of `PropertyName` to string
:   ​

`labels` array of string
:   ​

`channelConfig` [`ChannelConfig`](https://dev.nominal.io/api/legacy/ingest-service.md#channelconfig)
:   ​

`workspace` string (RID)
:   The workspace in which to create the dataset. If not provided, the dataset will be created in the default workspace for the user’s organization, if the default workspace for the organization is configured.

`markingRids` array of string (RID)
:   The markings to apply to the created dataset. If not provided, the dataset will be visible to all users in the same workspace.

### NewDataSource

`source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
:   ​

`properties` map of `PropertyName` to string
:   ​

`labels` array of string
:   ​

`description` string
:   ​

`name` string
:   ​

`timeColumnSpec` [`TimestampMetadata`](https://dev.nominal.io/api/legacy/ingest-service.md#timestampmetadata)
:   ​

`channelConfig` [`ChannelConfig`](https://dev.nominal.io/api/legacy/ingest-service.md#channelconfig)
:   ​

### NewVideoIngestDestination

`title` string
:   Title of the Video that will get created. If not provided, a name is deduced from the ingested file.

`description` string
:   Description that is applied to the newly created video

`properties` map of `PropertyName` to string
:   Key-Value properties that are applied to the newly created video

`labels` array of string
:   Labels that are applied to the newly created video

`videoFileDetails` [`VideoFileIngestDetails`](https://dev.nominal.io/api/legacy/ingest-service.md#videofileingestdetails)
:   Metadata to associate with any created video file

`workspace` string (RID)
:   The workspace in which to create the video. If not provided, the video will be created in the default workspace for the user’s organization, if the default workspace for the organization is configured.

`markingRids` array of string (RID)
:   The markings to apply to the created video. If not provided, the video will be visible to all users in the same workspace.

### ParquetOpts

Options for ingesting parquet files. Supported file formats include .parquet, .parquet.gz and archives such as .tar, .tar.gz, and .zip (must set the isArchive flag).

`source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
:   ​

`target` [`DatasetIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#datasetingesttarget) required
:   ​

`timestampMetadata` [`TimestampMetadata`](https://dev.nominal.io/api/legacy/ingest-service.md#timestampmetadata) required
:   ​

`channelPrefix` string required
:   ​

`channelNameOverrides` map of `ColumnName` to string
:   ​

`tagKeysFromColumns` array of string deprecated
:   **Deprecated:** Deprecated in favor of tagColumns.

`tagColumns` map of `TagName` to string
:   A map of tag names to column names to derive the tag values from.

`additionalFileTags` map of `TagName` to string
:   Specifies a tag set to apply to all data in the file.

`isArchive` boolean
:   If true, the file is an archive. Supported archive formats include .tar, .tar.gz, and .zip. Only files ending in .parquet within the archive will be ingested. If field not provided, defaults to false.

`excludeColumns` array of string
:   A set of column names to exclude from ingestion. These columns will not be ingested as channels. Useful for excluding columns that contain unsupported data types like multidimensional arrays.

### PointCloudIngestTarget

Set `type` to the name of one of the other fields, and set only that field.

`existing` [`ExistingSpatialIngestDestination`](https://dev.nominal.io/api/legacy/ingest-service.md#existingspatialingestdestination)
:   ​

### PointCloudMetadata

Metadata about the sensor that captured a point cloud.

`sensorModel` string
:   The sensor model name, e.g. “Velodyne VLP-16”, “Ouster OS1-128”.

`coordinateSystem` string
:   Coordinate reference system identifier, e.g. “EPSG:4326”, “ENU”.

`resolutionMm` number
:   Spatial resolution of the point cloud in millimeters.

`scanPattern` [`ScanPattern`](https://dev.nominal.io/api/legacy/ingest-service.md#scanpattern)
:   The scan pattern of the lidar sensor.

### PointCloudOpts

Options for ingesting point cloud data into Dagger via Nominal.

`source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
:   ​

`target` [`PointCloudIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#pointcloudingesttarget) required
:   ​

`daggerImportConfig` any JSON value required
:   The Dagger **v2** `ImportRequest` body — everything from that schema except `source_uri`, which Nominal fills in from the resolved presigned URL for `source`. Forwarded to Dagger’s `POST /v2/imports/{model_uuid}`. Construct via the language’s generated dagger-client types (e.g. `dagger_client.models.ImportRequest` in Python). Carries the required `archetype` (attribute definitions and sampler configuration), an optional `format` (the source format and its format-specific parameters, including the `geometry_type` and `columns` a CSV source needs; omit it to let Dagger infer the format from the file extension), and an optional `quiche_config` — all of which are domain knowledge the client owns when it produced the file. The legacy v1 shape, with `geometry_type` and `columns` at the top level, is rejected.

`sensorMetadata` [`PointCloudMetadata`](https://dev.nominal.io/api/legacy/ingest-service.md#pointcloudmetadata)
:   Sensor metadata to associate with the point cloud.

`channel` string required
:   Channel name for workbook integration.

`tags` map of `TagName` to string
:   Tags to apply to the point cloud series.

### PresignedFileIngestSource

`url` string required
:   ​

### ReingestFilesMapping

Maps the source dataset file ID to the destination dataset and file ID.

`fileMap` map of `DatasetFileId` to [`IngestDetails`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestdetails)
:   ​

### RelativeTimestamp

`timeUnit` [`TimeUnit`](https://dev.nominal.io/api/legacy/ingest-service.md#timeunit) required
:   ​

`offset` string (date-time)
:   Starting timestamp to use when indexing the file. This field is required when uploading a new file to an existing dataset.

### S3IngestSource

`path` string required
:   ​

### ScanPattern

`ROTATING`
:   ​

`SOLID_STATE`
:   ​

`FLASH`
:   ​

`MECHANICAL`
:   ​

### TimeOffsetSpec

Set `type` to the name of one of the other fields, and set only that field.

`nanos` [`Duration`](https://dev.nominal.io/api/legacy/ingest-service.md#duration)
:   ​

### Timestamp

Deprecated for new APIs. Use google.protobuf.Timestamp for absolute instants in proto APIs.

This legacy nanosecond precision timestamp type is represented by an epoch time in seconds and a nanosecond offset. The nanosecond offset is from the start of the epoch second, so must be less than 1 billion.

`seconds` integer (int64) required
:   ​

`nanos` integer (int64) required
:   ​

### TimestampManifest

The timestamp manifest files will contain a list of absolute timestamps, in nanoseconds, that correspond to each frame in a video. Each file should be of type JSON and store a single list, the length of which equals the number of frames in its corresponding video.

`sources` array of [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource)
:   ​

### TimestampMetadata

`seriesName` string required
:   ​

`timestampType` [`TimestampType`](https://dev.nominal.io/api/legacy/ingest-service.md#timestamptype) required
:   ​

### TimestampType

Set `type` to the name of one of the other fields, and set only that field.

`relative` [`RelativeTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#relativetimestamp)
:   ​

`absolute` [`AbsoluteTimestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#absolutetimestamp)
:   ​

### TimeUnit

`DAYS`
:   ​

`HOURS`
:   ​

`MINUTES`
:   ​

`SECONDS`
:   ​

`MILLISECONDS`
:   ​

`MICROSECONDS`
:   ​

`NANOSECONDS`
:   ​

### UtcTimestamp

`secondsSinceEpoch` integer (int64) required
:   ​

`offsetNanoseconds` integer (int64)
:   ​

### video.api.NoTimestampManifest

these values will not be updated after ingest time, to allow for resetting. The updated values are stored implicitly through the segment timestamps.

`startingTimestamp` [`Timestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#timestamp) required
:   Specifies the original starting timestamp of the video.

`scaleParameter` [`video.api.ScaleParameter`](https://dev.nominal.io/api/legacy/ingest-service.md#video-api-scaleparameter)
:   A field that specifies that the frame rate of the video does not match the frame rate of the camera | i.e. a slowed down or sped up video. Can specify either the camera frame rate or the absolute end time.

### video.api.ScaleParameter

Set `type` to the name of one of the other fields, and set only that field.

`trueFrameRate` number
:   ​

`endingTimestamp` [`Timestamp`](https://dev.nominal.io/api/legacy/ingest-service.md#timestamp)
:   ​

`scaleFactor` number
:   the scale factor can be used to calculate whether media duration differs from a video’s | real duration, and if so, the true frame rate of the camera. The video time will thus be scaled | by the ratio of the real duration to media duration, or media frame rate to true frame rate.

### VideoFileIngestDetails

`fileDescription` string
:   Description that is applied to the newly created video file. If not provided, defaults to the description used to create the video.

`fileProperties` map of `PropertyName` to string deprecated
:   Key-Value properties that are applied to the newly created video file. **Deprecated:** Field is ignored as video files do not have properties. Will be removed after 4/5/2025

`fileLabels` array of string deprecated
:   Labels that are applied to the newly created video file. **Deprecated:** Field is ignored as video files do not have labels. Will be removed after 4/5/2025

### VideoFileTimestampManifest

A video file’s timestamp manifest specifies how to determine absolute timestamps for each frame. In an MCAP file, this data is embedded within the source file. Alternatively, the manifest may be provided as an external sidecar file. If no manifest is available, the timestamps are calculated by applying a starting offset to the presentation timestamp of each frame.

Set `type` to the name of one of the other fields, and set only that field.

`mcap` [`McapTimestampManifest`](https://dev.nominal.io/api/legacy/ingest-service.md#mcaptimestampmanifest)
:   ​

`s3path` string
:   ​

`s3paths` array of string
:   Multiple sidecar manifest files whose per-frame timestamps are concatenated in list order. Used when a video’s frame timestamps are recorded across several files.

`noManifest` [`video.api.NoTimestampManifest`](https://dev.nominal.io/api/legacy/ingest-service.md#video-api-notimestampmanifest)
:   ​

### VideoIngestTarget

Set `type` to the name of one of the other fields, and set only that field.

`new` [`NewVideoIngestDestination`](https://dev.nominal.io/api/legacy/ingest-service.md#newvideoingestdestination)
:   ​

`existing` [`ExistingVideoIngestDestination`](https://dev.nominal.io/api/legacy/ingest-service.md#existingvideoingestdestination)
:   ​

### VideoOpts

`source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
:   ​

`target` [`VideoIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#videoingesttarget) required
:   ​

`timestampManifest` [`VideoFileTimestampManifest`](https://dev.nominal.io/api/legacy/ingest-service.md#videofiletimestampmanifest) required
:   ​

`overWriteSegments` boolean
:   If true, overlapping segments from other video files within the same video will be deleted before inserting new segments. The cached segment metadata for affected files will be recomputed.

### VideoOptsV2

`source` [`IngestSource`](https://dev.nominal.io/api/legacy/ingest-service.md#ingestsource) required
:   ​

`target` [`DatasetIngestTarget`](https://dev.nominal.io/api/legacy/ingest-service.md#datasetingesttarget) required
:   ​

`timestampManifest` [`VideoFileTimestampManifest`](https://dev.nominal.io/api/legacy/ingest-service.md#videofiletimestampmanifest) required
:   ​

`channel` string required
:   ​

`tags` map of `TagName` to string
:   ​

`overWriteSegments` boolean
:   If true, overlapping segments from other dataset files within the same series will be deleted before inserting new segments. The cached segment metadata for affected files will be recomputed.

### VideoTimestampManifest

Set `type` to the name of one of the other fields, and set only that field.

`noManifest` [`ingest.api.NoTimestampManifest`](https://dev.nominal.io/api/legacy/ingest-service.md#ingest-api-notimestampmanifest)
:   ​

`timestampManifests` [`TimestampManifest`](https://dev.nominal.io/api/legacy/ingest-service.md#timestampmanifest)
:   ​

Page design based on [Scalar](https://scalar.com), MIT License.