Ingest automation service

Creates and reports on ingest automations: server-side workflows that take an upload (one or more files) through asset/dataset/data-scope/run creation, ingest, and channel availability. Backs the upload wizard’s “Quick look” mode; the frontend creates the accompanying workbook. All endpoints validate that the target workspace is the sandbox workspace while the feature is gated. Resources are created as the requesting user.

OpenAPI file: IngestAutomationService.json

Endpoints

Create ingest automation

POST /ingest/v1/ingest-automations

Creates the automation and starts its workflow, returning immediately. Poll getIngestAutomation for progress. Throws WorkspaceNotSandbox outside the sandbox workspace while the feature is gated.

Body

workspaceRid string (RID) required

​

sources array of IngestSource

Locations of the already-uploaded files, all ingested into the same dataset. At least one; at most 20 (enforced by the handler). Only s3 sources are supported today.

s3

Set type to "s3".

s3 S3IngestSource

​

Fields of S3IngestSource

path string required

​

gcs

Set type to "gcs".

gcs GcsIngestSource

​

Fields of GcsIngestSource

path string required

​

presignedFile

Set type to "presignedFile".

presignedFile PresignedFileIngestSource

​

Fields of PresignedFileIngestSource

url string required

​

fileStore

Set type to "fileStore".

fileStore FileStoreIngestSource

​

Fields of FileStoreIngestSource

fileRevisionRid string (RID) required

​

timestampMetadata TimestampMetadata required

The timestamp column and interpretation the user confirmed in the wizard, applied to every file.

Fields of TimestampMetadata

seriesName string required

​

timestampType TimestampType required

​

relative

Set type to "relative".

relative RelativeTimestamp

​

Fields of RelativeTimestamp

timeUnit TimeUnit required

​

Values of TimeUnit

DAYS

​

HOURS

​

MINUTES

​

SECONDS

​

MILLISECONDS

​

MICROSECONDS

​

NANOSECONDS

​

offset string (date-time)

Starting timestamp to use when indexing the file. This field is required when uploading a new file to an existing dataset.

absolute

Set type to "absolute".

absolute AbsoluteTimestamp

​

iso8601

Set type to "iso8601".

iso8601 Iso8601Timestamp

​

epochOfTimeUnit

Set type to "epochOfTimeUnit".

epochOfTimeUnit EpochTimestamp

​

Fields of EpochTimestamp

timeUnit TimeUnit required

​

customFormat

Set type to "customFormat".

customFormat CustomTimestamp

​

Fields of CustomTimestamp

format string required

The format string should be in the format of the DateTimeFormatter class in Java.

defaultYear integer

Default year is accepted as an optional field for cases like IRIG time format and will be overridden by year in time format.

defaultDayOfYear integer

Default day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.

fileType IngestAutomationFileType

The format of every file in sources, ingested as wide-format data. Defaults to CSV when absent.

Values of IngestAutomationFileType

CSV

​

PARQUET

​

assetTitle string required

Maximum length 500 characters (enforced by the handler, like the other titles).

datasetTitle string required

Maximum length 500 characters.

runTitle string required

Maximum length 500 characters.

refName string required

Data scope name linking the created asset and dataset (typically “default”). Maximum length 100 characters.

Response

rid string (RID)

​

status IngestAutomationStatus

​

Values of IngestAutomationStatus

CREATING_RESOURCES

​

INGESTING

​

AWAITING_CHANNELS

​

COMPLETED

​

FAILED

​

workspaceRid string (RID)

​

createdByRid string (RID)

​

createdAt string (date-time)

​

updatedAt string (date-time)

​

assetRid string (RID)

Present once CREATING_RESOURCES has created the asset.

datasetRid string (RID)

Present once CREATING_RESOURCES has created the dataset.

runRid string (RID)

Present once CREATING_RESOURCES has created the run.

ingestJobRid string (RID)

Present once INGESTING has submitted the ingest job.

failedStep IngestAutomationStep

The step that terminated the automation. Present only for FAILED automations.

Values of IngestAutomationStep

CREATE_RESOURCES

​

SUBMIT_INGEST

​

AWAIT_INGEST

​

AWAIT_CHANNELS

​

errorMessage string

The failure detail recorded when the automation terminated. Present only for FAILED automations. May contain customer-derived content and is therefore not safe to log.

Request

curl

curl -X POST "https://api.gov.nominal.io/api/ingest/v1/ingest-automations" \
  -H "Authorization: Bearer <api key>" \
  -H "Content-Type: application/json" \
  --data '{
  "workspaceRid": "<workspace rid>",
  "timestampMetadata": {
    "seriesName": "<series name>",
    "timestampType": {
      "type": "relative",
      "relative": {
        "timeUnit": "DAYS"
      }
    }
  },
  "assetTitle": "<asset title>",
  "datasetTitle": "<dataset title>",
  "runTitle": "<run title>",
  "refName": "<ref name>"
}'

requests

import requests

response = requests.post(
    "https://api.gov.nominal.io/api/ingest/v1/ingest-automations",
    headers={"Authorization": "Bearer <api key>"},
    json={
        "workspaceRid": "<workspace rid>",
        "timestampMetadata": {
            "seriesName": "<series name>",
            "timestampType": {"type": "relative", "relative": {"timeUnit": "DAYS"}},
        },
        "assetTitle": "<asset title>",
        "datasetTitle": "<dataset title>",
        "runTitle": "<run title>",
        "refName": "<ref name>",
    },
)
response.raise_for_status()
print(response.json())

httpx

import httpx

response = httpx.post(
    "https://api.gov.nominal.io/api/ingest/v1/ingest-automations",
    headers={"Authorization": "Bearer <api key>"},
    json={
        "workspaceRid": "<workspace rid>",
        "timestampMetadata": {
            "seriesName": "<series name>",
            "timestampType": {"type": "relative", "relative": {"timeUnit": "DAYS"}},
        },
        "assetTitle": "<asset title>",
        "datasetTitle": "<dataset title>",
        "runTitle": "<run title>",
        "refName": "<ref name>",
    },
)
response.raise_for_status()
print(response.json())

aiohttp

import asyncio

import aiohttp


async def main() -> None:
    async with aiohttp.ClientSession() as session:
        async with session.post(
            "https://api.gov.nominal.io/api/ingest/v1/ingest-automations",
            headers={"Authorization": "Bearer <api key>"},
            json={
                "workspaceRid": "<workspace rid>",
                "timestampMetadata": {
                    "seriesName": "<series name>",
                    "timestampType": {
                        "type": "relative",
                        "relative": {"timeUnit": "DAYS"},
                    },
                },
                "assetTitle": "<asset title>",
                "datasetTitle": "<dataset title>",
                "runTitle": "<run title>",
                "refName": "<ref name>",
            },
        ) as response:
            response.raise_for_status()
            print(await response.json())


asyncio.run(main())

Response: 200

{
  "rid": "<ingest automation rid>",
  "status": "CREATING_RESOURCES",
  "workspaceRid": "<workspace rid>",
  "createdByRid": "<user rid>",
  "createdAt": "2026-01-01T00:00:00Z",
  "updatedAt": "2026-01-01T00:00:00Z",
  "assetRid": "<asset rid>",
  "datasetRid": "<dataset rid>",
  "runRid": "<run rid>",
  "ingestJobRid": "<ingest job rid>",
  "failedStep": "CREATE_RESOURCES",
  "errorMessage": "<error message>"
}

Get ingest automation

GET /ingest/v1/ingest-automations/{rid}

Returns the automation’s current status and the RIDs of every resource created so far.

Path parameters

rid string (RID) required

​

Response

rid string (RID)

​

status IngestAutomationStatus

​

Values of IngestAutomationStatus

CREATING_RESOURCES

​

INGESTING

​

AWAITING_CHANNELS

​

COMPLETED

​

FAILED

​

workspaceRid string (RID)

​

createdByRid string (RID)

​

createdAt string (date-time)

​

updatedAt string (date-time)

​

assetRid string (RID)

Present once CREATING_RESOURCES has created the asset.

datasetRid string (RID)

Present once CREATING_RESOURCES has created the dataset.

runRid string (RID)

Present once CREATING_RESOURCES has created the run.

ingestJobRid string (RID)

Present once INGESTING has submitted the ingest job.

failedStep IngestAutomationStep

The step that terminated the automation. Present only for FAILED automations.

Values of IngestAutomationStep

CREATE_RESOURCES

​

SUBMIT_INGEST

​

AWAIT_INGEST

​

AWAIT_CHANNELS

​

errorMessage string

The failure detail recorded when the automation terminated. Present only for FAILED automations. May contain customer-derived content and is therefore not safe to log.

Request

curl

curl "https://api.gov.nominal.io/api/ingest/v1/ingest-automations/<rid>" \
  -H "Authorization: Bearer <api key>"

requests

import requests

response = requests.get(
    "https://api.gov.nominal.io/api/ingest/v1/ingest-automations/<rid>",
    headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())

httpx

import httpx

response = httpx.get(
    "https://api.gov.nominal.io/api/ingest/v1/ingest-automations/<rid>",
    headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())

aiohttp

import asyncio

import aiohttp


async def main() -> None:
    async with aiohttp.ClientSession() as session:
        async with session.get(
            "https://api.gov.nominal.io/api/ingest/v1/ingest-automations/<rid>",
            headers={"Authorization": "Bearer <api key>"},
        ) as response:
            response.raise_for_status()
            print(await response.json())


asyncio.run(main())

Response: 200

{
  "rid": "<ingest automation rid>",
  "status": "CREATING_RESOURCES",
  "workspaceRid": "<workspace rid>",
  "createdByRid": "<user rid>",
  "createdAt": "2026-01-01T00:00:00Z",
  "updatedAt": "2026-01-01T00:00:00Z",
  "assetRid": "<asset rid>",
  "datasetRid": "<dataset rid>",
  "runRid": "<run rid>",
  "ingestJobRid": "<ingest job rid>",
  "failedStep": "CREATE_RESOURCES",
  "errorMessage": "<error message>"
}

Errors

Errors shows how these errors are sent.

Each error’s errorName is IngestAutomation: followed by its name below.

Status	Name	Notes
400	WorkspaceNotSandbox	Ingest automations are temporarily restricted to the sandbox workspace while the “Quick look” upload mode is gated (see the Quick look upon ingest RFC). Lifting the restriction removes this validation.
404	IngestAutomationNotFound

Types

AbsoluteTimestamp

Set type to the name of one of the other fields, and set only that field.

iso8601 Iso8601Timestamp

​

epochOfTimeUnit EpochTimestamp

​

customFormat CustomTimestamp

​

CustomTimestamp

format string required

The format string should be in the format of the DateTimeFormatter class in Java.

defaultYear integer

Default year is accepted as an optional field for cases like IRIG time format and will be overridden by year in time format.

defaultDayOfYear integer

Default day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.

EpochTimestamp

timeUnit TimeUnit required

​

FileStoreIngestSource

A resolved File Store revision to ingest, whose bytes are read in place from File Store storage.

fileRevisionRid string (RID) required

​

GcsIngestSource

path string required

​

IngestAutomationFileType

The format of the uploaded files. Every file in one automation shares it.

CSV

​

PARQUET

​

IngestAutomationStatus

Progress of an ingest automation, in execution order. Statuses map one-to-one onto the client’s progress checklist. Resources created before a FAILED terminal state persist (the flow is intentionally non-atomic, matching the existing ingest flow).

CREATING_RESOURCES

​

INGESTING

​

AWAITING_CHANNELS

​

COMPLETED

​

FAILED

​

IngestAutomationStep

The individual workflow steps, used to report where a FAILED automation stopped.

CREATE_RESOURCES

​

SUBMIT_INGEST

​

AWAIT_INGEST

​

AWAIT_CHANNELS

​

IngestSource

Set type to the name of one of the other fields, and set only that field.

s3 S3IngestSource

​

gcs GcsIngestSource

​

presignedFile PresignedFileIngestSource

​

fileStore FileStoreIngestSource

​

Iso8601Timestamp

An empty JSON object, {}.

PresignedFileIngestSource

url string required

​

RelativeTimestamp

timeUnit TimeUnit required

​

offset string (date-time)

Starting timestamp to use when indexing the file. This field is required when uploading a new file to an existing dataset.

S3IngestSource

path string required

​

TimestampMetadata

seriesName string required

​

timestampType TimestampType required

​

TimestampType

Set type to the name of one of the other fields, and set only that field.

relative RelativeTimestamp

​

absolute AbsoluteTimestamp

​

TimeUnit

DAYS

​

HOURS

​

MINUTES

​

SECONDS

​

MILLISECONDS

​

MICROSECONDS

​

NANOSECONDS

​

Page design based on Scalar, MIT License.