Containerized extractor service#

The Containerized Extractor service provides functionality for creating, updating, and archiving containerized extractors.

OpenAPI file: ContainerizedExtractorService.json

Endpoints#

Register containerized extractor#

POST /extractors/v1/container

Registers a containerized extractor for a given set of containerized files.

Body

name string required

​

description string

​

image DockerImageSource deprecated

Deprecated: use containerImageRid instead for self-hosted images. Container image for the extractor. Used for external registries. Ignored when containerImageRid is set. Deprecated: Use containerImageRid for self-hosted images.

Fields of DockerImageSource
registry string required

The container registry where the image is hosted.

repository string required

The repository name of the image.

tagDetails TagDetails required

​

Fields of TagDetails
tags array of string

​

defaultTag string required

​

authentication Authentication required

Optional authentication for accessing private container registries.

Set type to "userAndPassword".

userAndPassword UserAndPasswordAuthentication

​

Fields of UserAndPasswordAuthentication
username string required

Username for registry authentication.

passwordSecretRid string (RID) required

The RID of the secret containing the password for registry authentication.

Set type to "public".

public PublicAuthentication

​

command string

​

containerImageRid string (RID)

RID of a self-hosted container image. When set, the extractor uses the internal registry and the image field is ignored.

inputs array of FileExtractionInput

payload must match input defined in containerized extraction

Fields of FileExtractionInput
environmentVariable string required

The environment variable that stores the path to the input file.

name string required

Name of the input file which users will be prompted with

description string

Description of the input file which users will be prompted with

fileFilters array of FileFilter

Optionally filter files for file selection

Set type to "suffix".

suffix string

​

required boolean

Whether the input file is required for the extractor to run.

parameters array of FileExtractionParameter

Describes the parameters of the extractor.

Fields of FileExtractionParameter
environmentVariable string required

The environment variable that stores the argument

name string required

Name of the parameter which users will be prompted with

description string

Description of the parameter which users will be prompted with

required boolean

Whether the parameter is required for the extractor to run.

properties map of PropertyName to string

​

labels array of string

​

workspace string (RID) required

The workspace in which to create the extractor

timestampMetadata TimestampMetadata

Metadata about the intermediate parquet this extractor will produce. If not set, timestamp metadata must be provided at ingest time.

Fields of TimestampMetadata
seriesName string required

​

timestampType TimestampType required

​

Set type to "relative".

relative RelativeTimestamp

​

Fields of RelativeTimestamp
timeUnit TimeUnit required

​

Values of TimeUnit
DAYS

​

HOURS

​

MINUTES

​

SECONDS

​

MILLISECONDS

​

MICROSECONDS

​

NANOSECONDS

​

offset string (date-time)

Starting timestamp to use when indexing the file. This field is required when uploading a new file to an existing dataset.

Set type to "absolute".

absolute AbsoluteTimestamp

​

Set type to "iso8601".

iso8601 Iso8601Timestamp

​

Set type to "epochOfTimeUnit".

epochOfTimeUnit EpochTimestamp

​

Fields of EpochTimestamp
timeUnit TimeUnit required

​

Set type to "customFormat".

customFormat CustomTimestamp

​

Fields of CustomTimestamp
format string required

The format string should be in the format of the DateTimeFormatter class in Java.

defaultYear integer

Default year is accepted as an optional field for cases like IRIG time format and will be overridden by year in time format.

defaultDayOfYear integer

Default day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.

outputFileFormat FileOutputFormat

The format of the output file. Currently only “parquet”, “csv”, “parquet.tar” are supported

Values of FileOutputFormat
PARQUET

​

CSV

​

PARQUET_TAR

​

AVRO_STREAM

​

JSON_L

​

MANIFEST

​

Response

extractorRid string (RID)

​

Request

curl -X POST "https://api.gov.nominal.io/api/extractors/v1/container" \
  -H "Authorization: Bearer <api key>" \
  -H "Content-Type: application/json" \
  --data '{
  "name": "<name>",
  "workspace": "<workspace rid>"
}'

import requests

response = requests.post(
    "https://api.gov.nominal.io/api/extractors/v1/container",
    headers={"Authorization": "Bearer <api key>"},
    json={"name": "<name>", "workspace": "<workspace rid>"},
)
response.raise_for_status()
print(response.json())

import httpx

response = httpx.post(
    "https://api.gov.nominal.io/api/extractors/v1/container",
    headers={"Authorization": "Bearer <api key>"},
    json={"name": "<name>", "workspace": "<workspace rid>"},
)
response.raise_for_status()
print(response.json())

import asyncio

import aiohttp


async def main() -> None:
    async with aiohttp.ClientSession() as session:
        async with session.post(
            "https://api.gov.nominal.io/api/extractors/v1/container",
            headers={"Authorization": "Bearer <api key>"},
            json={"name": "<name>", "workspace": "<workspace rid>"},
        ) as response:
            response.raise_for_status()
            print(await response.json())


asyncio.run(main())

Response: 200

{
  "extractorRid": "<containerized extractor rid>"
}

Get containerized extractor#

GET /extractors/v1/container/{extractorRid}

Get a containerized extractor by its RID.

Path parameters

extractorRid string (RID) required

​

Response

rid string (RID)

Unique resource identifier for the extractor.

name string

The name of the extractor as defined by the user.

description string

Optional description of the extractor.

image DockerImageSource deprecated

Deprecated: use containerImageRid instead for self-hosted images. Container image used to run the extractor via an external registry. When containerImageRid is set, this field contains placeholder values and should be ignored. Deprecated: Use containerImageRid for self-hosted images.

Fields of DockerImageSource
registry string

The container registry where the image is hosted.

repository string

The repository name of the image.

tagDetails TagDetails

​

Fields of TagDetails
tags array of string

​

defaultTag string

​

authentication Authentication

Optional authentication for accessing private container registries.

Set type to "userAndPassword".

userAndPassword UserAndPasswordAuthentication

​

Fields of UserAndPasswordAuthentication
username string

Username for registry authentication.

passwordSecretRid string (RID)

The RID of the secret containing the password for registry authentication.

Set type to "public".

public PublicAuthentication

​

command string

​

containerImageRid string (RID)

RID of a self-hosted container image in the internal registry. When set, the extractor uses the internal registry; when absent, the legacy image field is used.

inputs array of FileExtractionInput

The input files that this extractor requires, mapped to environment variables that store the path to the file.

Fields of FileExtractionInput
environmentVariable string

The environment variable that stores the path to the input file.

name string

Name of the input file which users will be prompted with

description string

Description of the input file which users will be prompted with

fileFilters array of FileFilter

Optionally filter files for file selection

Set type to "suffix".

suffix string

​

required boolean

Whether the input file is required for the extractor to run.

parameters array of FileExtractionParameter

Describes the parameters of the extractor.

Fields of FileExtractionParameter
environmentVariable string

The environment variable that stores the argument

name string

Name of the parameter which users will be prompted with

description string

Description of the parameter which users will be prompted with

required boolean

Whether the parameter is required for the extractor to run.

properties map of PropertyName to string

Additional properties associated with this extractor.

labels array of string

Set of labels applied to this extractor.

createdAt string (date-time)

Timestamp when this extractor was created.

isArchived boolean

Whether this extractor is archived.

timestampMetadata TimestampMetadata

Metadata about the intermediate parquet this extractor will produce. If not set, timestamp metadata must be provided at ingest time.

Fields of TimestampMetadata
seriesName string

​

timestampType TimestampType

​

Set type to "relative".

relative RelativeTimestamp

​

Fields of RelativeTimestamp
timeUnit TimeUnit

​

Values of TimeUnit
DAYS

​

HOURS

​

MINUTES

​

SECONDS

​

MILLISECONDS

​

MICROSECONDS

​

NANOSECONDS

​

offset string (date-time)

Starting timestamp to use when indexing the file. This field is required when uploading a new file to an existing dataset.

Set type to "absolute".

absolute AbsoluteTimestamp

​

Set type to "iso8601".

iso8601 Iso8601Timestamp

​

Set type to "epochOfTimeUnit".

epochOfTimeUnit EpochTimestamp

​

Fields of EpochTimestamp
timeUnit TimeUnit

​

Set type to "customFormat".

customFormat CustomTimestamp

​

Fields of CustomTimestamp
format string

The format string should be in the format of the DateTimeFormatter class in Java.

defaultYear integer

Default year is accepted as an optional field for cases like IRIG time format and will be overridden by year in time format.

defaultDayOfYear integer

Default day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.

outputFileFormat FileOutputFormat

The format of the output file. Currently only “parquet”, “csv”, “parquet.tar” are supported

Values of FileOutputFormat
PARQUET

​

CSV

​

PARQUET_TAR

​

AVRO_STREAM

​

JSON_L

​

MANIFEST

​

exitCodeMappings array of ExitCodeMapping

Fallback exit-code-to-error mappings for extractors that cannot emit a structured termination-log document. Populated from the active container image for v2 extractors; empty otherwise.

Fields of ExitCodeMapping
exitCode integer

The container process exit code this mapping applies to.

code string

Machine-readable error code. Extractor-defined codes must fall outside the reserved platform namespace.

message string

Human-readable error message.

retryable boolean

Whether the platform may retry the ingest for this failure.

resources ContainerResources

Compute resources declared on the extractor’s container image. Read-only: set at image build time and never accepted on register or update. Absent, and absent per field, for an image that declares none, in which case the deployment-wide default configured on combined-service applies.

Fields of ContainerResources
cpuCores integer

CPU cores requested for the extractor container.

memoryGib integer

Memory in GiB, applied as both the request and the limit.

diskGib integer

Size in GiB of each of the ephemeral input and output volumes.

Request

curl "https://api.gov.nominal.io/api/extractors/v1/container/<extractor rid>" \
  -H "Authorization: Bearer <api key>"

import requests

response = requests.get(
    "https://api.gov.nominal.io/api/extractors/v1/container/<extractor rid>",
    headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())

import httpx

response = httpx.get(
    "https://api.gov.nominal.io/api/extractors/v1/container/<extractor rid>",
    headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())

import asyncio

import aiohttp


async def main() -> None:
    async with aiohttp.ClientSession() as session:
        async with session.get(
            "https://api.gov.nominal.io/api/extractors/v1/container/<extractor rid>",
            headers={"Authorization": "Bearer <api key>"},
        ) as response:
            response.raise_for_status()
            print(await response.json())


asyncio.run(main())

Response: 200

{
  "rid": "<containerized extractor rid>",
  "name": "<name>",
  "description": "<description>",
  "image": {
    "registry": "<registry>",
    "repository": "<repository>",
    "tagDetails": {
      "tags": [
        "<tags>"
      ],
      "defaultTag": "<default tag>"
    },
    "authentication": {
      "type": "userAndPassword",
      "userAndPassword": {
        "username": "<username>",
        "passwordSecretRid": "<secret rid>"
      }
    },
    "command": "<command>"
  },
  "containerImageRid": "<container image rid>",
  "inputs": [
    {
      "environmentVariable": "<environment variable>",
      "name": "<name>",
      "description": "<description>",
      "fileFilters": [
        {
          "type": "suffix",
          "suffix": "<file suffix>"
        }
      ],
      "required": true
    }
  ],
  "parameters": [
    {
      "environmentVariable": "<environment variable>",
      "name": "<name>",
      "description": "<description>",
      "required": true
    }
  ],
  "properties": {
    "<key>": "<property value>"
  },
  "labels": [
    "<label>"
  ],
  "createdAt": "2026-01-01T00:00:00Z",
  "isArchived": true,
  "timestampMetadata": {
    "seriesName": "<series name>",
    "timestampType": {
      "type": "relative",
      "relative": {
        "timeUnit": "DAYS",
        "offset": "2026-01-01T00:00:00Z"
      }
    }
  },
  "outputFileFormat": "PARQUET",
  "exitCodeMappings": [
    {
      "exitCode": 1,
      "code": "<code>",
      "message": "<message>",
      "retryable": true
    }
  ],
  "resources": {
    "cpuCores": 1,
    "memoryGib": 1,
    "diskGib": 1
  }
}

Update containerized extractor#

PUT /extractors/v1/container/{extractorRid}

Update a containerized extractor. Only the fields that are set in the request will be updated.

Path parameters

extractorRid string (RID) required

​

Body

name string

​

description string

​

inputs array of FileExtractionInput

​

Fields of FileExtractionInput
environmentVariable string required

The environment variable that stores the path to the input file.

name string required

Name of the input file which users will be prompted with

description string

Description of the input file which users will be prompted with

fileFilters array of FileFilter

Optionally filter files for file selection

Set type to "suffix".

suffix string

​

required boolean

Whether the input file is required for the extractor to run.

parameters array of FileExtractionParameter

​

Fields of FileExtractionParameter
environmentVariable string required

The environment variable that stores the argument

name string required

Name of the parameter which users will be prompted with

description string

Description of the parameter which users will be prompted with

required boolean

Whether the parameter is required for the extractor to run.

properties map of PropertyName to string

​

labels array of string

​

timestampMetadata TimestampMetadata

​

Fields of TimestampMetadata
seriesName string required

​

timestampType TimestampType required

​

Set type to "relative".

relative RelativeTimestamp

​

Fields of RelativeTimestamp
timeUnit TimeUnit required

​

Values of TimeUnit
DAYS

​

HOURS

​

MINUTES

​

SECONDS

​

MILLISECONDS

​

MICROSECONDS

​

NANOSECONDS

​

offset string (date-time)

Starting timestamp to use when indexing the file. This field is required when uploading a new file to an existing dataset.

Set type to "absolute".

absolute AbsoluteTimestamp

​

Set type to "iso8601".

iso8601 Iso8601Timestamp

​

Set type to "epochOfTimeUnit".

epochOfTimeUnit EpochTimestamp

​

Fields of EpochTimestamp
timeUnit TimeUnit required

​

Set type to "customFormat".

customFormat CustomTimestamp

​

Fields of CustomTimestamp
format string required

The format string should be in the format of the DateTimeFormatter class in Java.

defaultYear integer

Default year is accepted as an optional field for cases like IRIG time format and will be overridden by year in time format.

defaultDayOfYear integer

Default day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.

outputFileFormat FileOutputFormat

​

Values of FileOutputFormat
PARQUET

​

CSV

​

PARQUET_TAR

​

AVRO_STREAM

​

JSON_L

​

MANIFEST

​

registry string

​

repository string

​

tags array of string

​

defaultTag string

​

authentication Authentication

​

Set type to "userAndPassword".

userAndPassword UserAndPasswordAuthentication

​

Fields of UserAndPasswordAuthentication
username string required

Username for registry authentication.

passwordSecretRid string (RID) required

The RID of the secret containing the password for registry authentication.

Set type to "public".

public PublicAuthentication

​

command string

​

Response

rid string (RID)

Unique resource identifier for the extractor.

name string

The name of the extractor as defined by the user.

description string

Optional description of the extractor.

image DockerImageSource deprecated

Deprecated: use containerImageRid instead for self-hosted images. Container image used to run the extractor via an external registry. When containerImageRid is set, this field contains placeholder values and should be ignored. Deprecated: Use containerImageRid for self-hosted images.

Fields of DockerImageSource
registry string

The container registry where the image is hosted.

repository string

The repository name of the image.

tagDetails TagDetails

​

Fields of TagDetails
tags array of string

​

defaultTag string

​

authentication Authentication

Optional authentication for accessing private container registries.

Set type to "userAndPassword".

userAndPassword UserAndPasswordAuthentication

​

Fields of UserAndPasswordAuthentication
username string

Username for registry authentication.

passwordSecretRid string (RID)

The RID of the secret containing the password for registry authentication.

Set type to "public".

public PublicAuthentication

​

command string

​

containerImageRid string (RID)

RID of a self-hosted container image in the internal registry. When set, the extractor uses the internal registry; when absent, the legacy image field is used.

inputs array of FileExtractionInput

The input files that this extractor requires, mapped to environment variables that store the path to the file.

Fields of FileExtractionInput
environmentVariable string

The environment variable that stores the path to the input file.

name string

Name of the input file which users will be prompted with

description string

Description of the input file which users will be prompted with

fileFilters array of FileFilter

Optionally filter files for file selection

Set type to "suffix".

suffix string

​

required boolean

Whether the input file is required for the extractor to run.

parameters array of FileExtractionParameter

Describes the parameters of the extractor.

Fields of FileExtractionParameter
environmentVariable string

The environment variable that stores the argument

name string

Name of the parameter which users will be prompted with

description string

Description of the parameter which users will be prompted with

required boolean

Whether the parameter is required for the extractor to run.

properties map of PropertyName to string

Additional properties associated with this extractor.

labels array of string

Set of labels applied to this extractor.

createdAt string (date-time)

Timestamp when this extractor was created.

isArchived boolean

Whether this extractor is archived.

timestampMetadata TimestampMetadata

Metadata about the intermediate parquet this extractor will produce. If not set, timestamp metadata must be provided at ingest time.

Fields of TimestampMetadata
seriesName string

​

timestampType TimestampType

​

Set type to "relative".

relative RelativeTimestamp

​

Fields of RelativeTimestamp
timeUnit TimeUnit

​

Values of TimeUnit
DAYS

​

HOURS

​

MINUTES

​

SECONDS

​

MILLISECONDS

​

MICROSECONDS

​

NANOSECONDS

​

offset string (date-time)

Starting timestamp to use when indexing the file. This field is required when uploading a new file to an existing dataset.

Set type to "absolute".

absolute AbsoluteTimestamp

​

Set type to "iso8601".

iso8601 Iso8601Timestamp

​

Set type to "epochOfTimeUnit".

epochOfTimeUnit EpochTimestamp

​

Fields of EpochTimestamp
timeUnit TimeUnit

​

Set type to "customFormat".

customFormat CustomTimestamp

​

Fields of CustomTimestamp
format string

The format string should be in the format of the DateTimeFormatter class in Java.

defaultYear integer

Default year is accepted as an optional field for cases like IRIG time format and will be overridden by year in time format.

defaultDayOfYear integer

Default day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.

outputFileFormat FileOutputFormat

The format of the output file. Currently only “parquet”, “csv”, “parquet.tar” are supported

Values of FileOutputFormat
PARQUET

​

CSV

​

PARQUET_TAR

​

AVRO_STREAM

​

JSON_L

​

MANIFEST

​

exitCodeMappings array of ExitCodeMapping

Fallback exit-code-to-error mappings for extractors that cannot emit a structured termination-log document. Populated from the active container image for v2 extractors; empty otherwise.

Fields of ExitCodeMapping
exitCode integer

The container process exit code this mapping applies to.

code string

Machine-readable error code. Extractor-defined codes must fall outside the reserved platform namespace.

message string

Human-readable error message.

retryable boolean

Whether the platform may retry the ingest for this failure.

resources ContainerResources

Compute resources declared on the extractor’s container image. Read-only: set at image build time and never accepted on register or update. Absent, and absent per field, for an image that declares none, in which case the deployment-wide default configured on combined-service applies.

Fields of ContainerResources
cpuCores integer

CPU cores requested for the extractor container.

memoryGib integer

Memory in GiB, applied as both the request and the limit.

diskGib integer

Size in GiB of each of the ephemeral input and output volumes.

Request

curl -X PUT "https://api.gov.nominal.io/api/extractors/v1/container/<extractor rid>" \
  -H "Authorization: Bearer <api key>" \
  -H "Content-Type: application/json" \
  --data '{
  "name": "<name>",
  "description": "<description>",
  "inputs": [
    {
      "environmentVariable": "<environment variable>",
      "name": "<name>"
    }
  ],
  "parameters": [
    {
      "environmentVariable": "<environment variable>",
      "name": "<name>"
    }
  ],
  "properties": {
    "<key>": "<property value>"
  },
  "labels": [
    "<label>"
  ],
  "timestampMetadata": {
    "seriesName": "<series name>",
    "timestampType": {
      "type": "relative",
      "relative": {
        "timeUnit": "DAYS"
      }
    }
  },
  "outputFileFormat": "PARQUET",
  "registry": "<registry>",
  "repository": "<repository>",
  "tags": [
    "<tags>"
  ],
  "defaultTag": "<default tag>",
  "authentication": {
    "type": "userAndPassword",
    "userAndPassword": {
      "username": "<username>",
      "passwordSecretRid": "<secret rid>"
    }
  },
  "command": "<command>"
}'

import requests

response = requests.put(
    "https://api.gov.nominal.io/api/extractors/v1/container/<extractor rid>",
    headers={"Authorization": "Bearer <api key>"},
    json={
        "name": "<name>",
        "description": "<description>",
        "inputs": [{"environmentVariable": "<environment variable>", "name": "<name>"}],
        "parameters": [
            {"environmentVariable": "<environment variable>", "name": "<name>"},
        ],
        "properties": {"<key>": "<property value>"},
        "labels": ["<label>"],
        "timestampMetadata": {
            "seriesName": "<series name>",
            "timestampType": {"type": "relative", "relative": {"timeUnit": "DAYS"}},
        },
        "outputFileFormat": "PARQUET",
        "registry": "<registry>",
        "repository": "<repository>",
        "tags": ["<tags>"],
        "defaultTag": "<default tag>",
        "authentication": {
            "type": "userAndPassword",
            "userAndPassword": {
                "username": "<username>",
                "passwordSecretRid": "<secret rid>",
            },
        },
        "command": "<command>",
    },
)
response.raise_for_status()
print(response.json())

import httpx

response = httpx.put(
    "https://api.gov.nominal.io/api/extractors/v1/container/<extractor rid>",
    headers={"Authorization": "Bearer <api key>"},
    json={
        "name": "<name>",
        "description": "<description>",
        "inputs": [{"environmentVariable": "<environment variable>", "name": "<name>"}],
        "parameters": [
            {"environmentVariable": "<environment variable>", "name": "<name>"},
        ],
        "properties": {"<key>": "<property value>"},
        "labels": ["<label>"],
        "timestampMetadata": {
            "seriesName": "<series name>",
            "timestampType": {"type": "relative", "relative": {"timeUnit": "DAYS"}},
        },
        "outputFileFormat": "PARQUET",
        "registry": "<registry>",
        "repository": "<repository>",
        "tags": ["<tags>"],
        "defaultTag": "<default tag>",
        "authentication": {
            "type": "userAndPassword",
            "userAndPassword": {
                "username": "<username>",
                "passwordSecretRid": "<secret rid>",
            },
        },
        "command": "<command>",
    },
)
response.raise_for_status()
print(response.json())

import asyncio

import aiohttp


async def main() -> None:
    async with aiohttp.ClientSession() as session:
        async with session.put(
            "https://api.gov.nominal.io/api/extractors/v1/container/<extractor rid>",
            headers={"Authorization": "Bearer <api key>"},
            json={
                "name": "<name>",
                "description": "<description>",
                "inputs": [
                    {"environmentVariable": "<environment variable>", "name": "<name>"},
                ],
                "parameters": [
                    {"environmentVariable": "<environment variable>", "name": "<name>"},
                ],
                "properties": {"<key>": "<property value>"},
                "labels": ["<label>"],
                "timestampMetadata": {
                    "seriesName": "<series name>",
                    "timestampType": {
                        "type": "relative",
                        "relative": {"timeUnit": "DAYS"},
                    },
                },
                "outputFileFormat": "PARQUET",
                "registry": "<registry>",
                "repository": "<repository>",
                "tags": ["<tags>"],
                "defaultTag": "<default tag>",
                "authentication": {
                    "type": "userAndPassword",
                    "userAndPassword": {
                        "username": "<username>",
                        "passwordSecretRid": "<secret rid>",
                    },
                },
                "command": "<command>",
            },
        ) as response:
            response.raise_for_status()
            print(await response.json())


asyncio.run(main())

Response: 200

{
  "rid": "<containerized extractor rid>",
  "name": "<name>",
  "description": "<description>",
  "image": {
    "registry": "<registry>",
    "repository": "<repository>",
    "tagDetails": {
      "tags": [
        "<tags>"
      ],
      "defaultTag": "<default tag>"
    },
    "authentication": {
      "type": "userAndPassword",
      "userAndPassword": {
        "username": "<username>",
        "passwordSecretRid": "<secret rid>"
      }
    },
    "command": "<command>"
  },
  "containerImageRid": "<container image rid>",
  "inputs": [
    {
      "environmentVariable": "<environment variable>",
      "name": "<name>",
      "description": "<description>",
      "fileFilters": [
        {
          "type": "suffix",
          "suffix": "<file suffix>"
        }
      ],
      "required": true
    }
  ],
  "parameters": [
    {
      "environmentVariable": "<environment variable>",
      "name": "<name>",
      "description": "<description>",
      "required": true
    }
  ],
  "properties": {
    "<key>": "<property value>"
  },
  "labels": [
    "<label>"
  ],
  "createdAt": "2026-01-01T00:00:00Z",
  "isArchived": true,
  "timestampMetadata": {
    "seriesName": "<series name>",
    "timestampType": {
      "type": "relative",
      "relative": {
        "timeUnit": "DAYS",
        "offset": "2026-01-01T00:00:00Z"
      }
    }
  },
  "outputFileFormat": "PARQUET",
  "exitCodeMappings": [
    {
      "exitCode": 1,
      "code": "<code>",
      "message": "<message>",
      "retryable": true
    }
  ],
  "resources": {
    "cpuCores": 1,
    "memoryGib": 1,
    "diskGib": 1
  }
}

Search containerized extractors#

POST /extractors/v1/container/search

Search for containerized extractors based on query parameters.

Body

query SearchContainerizedExtractorsQuery required

​

Set type to "searchText".

searchText string

​

Set type to "label".

label string

​

Set type to "property".

property Property

​

Fields of Property
name string required

​

value string required

​

Set type to "and".

and array of SearchContainerizedExtractorsQuery

​

Set type to "or".

or array of SearchContainerizedExtractorsQuery

​

Set type to "workspace".

workspace string (RID)

​

Response

The body is an array of ContainerizedExtractor.

Fields of ContainerizedExtractor
rid string (RID)

Unique resource identifier for the extractor.

name string

The name of the extractor as defined by the user.

description string

Optional description of the extractor.

image DockerImageSource deprecated

Deprecated: use containerImageRid instead for self-hosted images. Container image used to run the extractor via an external registry. When containerImageRid is set, this field contains placeholder values and should be ignored. Deprecated: Use containerImageRid for self-hosted images.

Fields of DockerImageSource
registry string

The container registry where the image is hosted.

repository string

The repository name of the image.

tagDetails TagDetails

​

Fields of TagDetails
tags array of string

​

defaultTag string

​

authentication Authentication

Optional authentication for accessing private container registries.

Set type to "userAndPassword".

userAndPassword UserAndPasswordAuthentication

​

Fields of UserAndPasswordAuthentication
username string

Username for registry authentication.

passwordSecretRid string (RID)

The RID of the secret containing the password for registry authentication.

Set type to "public".

public PublicAuthentication

​

command string

​

containerImageRid string (RID)

RID of a self-hosted container image in the internal registry. When set, the extractor uses the internal registry; when absent, the legacy image field is used.

inputs array of FileExtractionInput

The input files that this extractor requires, mapped to environment variables that store the path to the file.

Fields of FileExtractionInput
environmentVariable string

The environment variable that stores the path to the input file.

name string

Name of the input file which users will be prompted with

description string

Description of the input file which users will be prompted with

fileFilters array of FileFilter

Optionally filter files for file selection

Set type to "suffix".

suffix string

​

required boolean

Whether the input file is required for the extractor to run.

parameters array of FileExtractionParameter

Describes the parameters of the extractor.

Fields of FileExtractionParameter
environmentVariable string

The environment variable that stores the argument

name string

Name of the parameter which users will be prompted with

description string

Description of the parameter which users will be prompted with

required boolean

Whether the parameter is required for the extractor to run.

properties map of PropertyName to string

Additional properties associated with this extractor.

labels array of string

Set of labels applied to this extractor.

createdAt string (date-time)

Timestamp when this extractor was created.

isArchived boolean

Whether this extractor is archived.

timestampMetadata TimestampMetadata

Metadata about the intermediate parquet this extractor will produce. If not set, timestamp metadata must be provided at ingest time.

Fields of TimestampMetadata
seriesName string

​

timestampType TimestampType

​

Set type to "relative".

relative RelativeTimestamp

​

Fields of RelativeTimestamp
timeUnit TimeUnit

​

Values of TimeUnit
DAYS

​

HOURS

​

MINUTES

​

SECONDS

​

MILLISECONDS

​

MICROSECONDS

​

NANOSECONDS

​

offset string (date-time)

Starting timestamp to use when indexing the file. This field is required when uploading a new file to an existing dataset.

Set type to "absolute".

absolute AbsoluteTimestamp

​

Set type to "iso8601".

iso8601 Iso8601Timestamp

​

Set type to "epochOfTimeUnit".

epochOfTimeUnit EpochTimestamp

​

Fields of EpochTimestamp
timeUnit TimeUnit

​

Set type to "customFormat".

customFormat CustomTimestamp

​

Fields of CustomTimestamp
format string

The format string should be in the format of the DateTimeFormatter class in Java.

defaultYear integer

Default year is accepted as an optional field for cases like IRIG time format and will be overridden by year in time format.

defaultDayOfYear integer

Default day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.

outputFileFormat FileOutputFormat

The format of the output file. Currently only “parquet”, “csv”, “parquet.tar” are supported

Values of FileOutputFormat
PARQUET

​

CSV

​

PARQUET_TAR

​

AVRO_STREAM

​

JSON_L

​

MANIFEST

​

exitCodeMappings array of ExitCodeMapping

Fallback exit-code-to-error mappings for extractors that cannot emit a structured termination-log document. Populated from the active container image for v2 extractors; empty otherwise.

Fields of ExitCodeMapping
exitCode integer

The container process exit code this mapping applies to.

code string

Machine-readable error code. Extractor-defined codes must fall outside the reserved platform namespace.

message string

Human-readable error message.

retryable boolean

Whether the platform may retry the ingest for this failure.

resources ContainerResources

Compute resources declared on the extractor’s container image. Read-only: set at image build time and never accepted on register or update. Absent, and absent per field, for an image that declares none, in which case the deployment-wide default configured on combined-service applies.

Fields of ContainerResources
cpuCores integer

CPU cores requested for the extractor container.

memoryGib integer

Memory in GiB, applied as both the request and the limit.

diskGib integer

Size in GiB of each of the ephemeral input and output volumes.

Request

curl -X POST "https://api.gov.nominal.io/api/extractors/v1/container/search" \
  -H "Authorization: Bearer <api key>" \
  -H "Content-Type: application/json" \
  --data '{
  "query": {
    "type": "searchText",
    "searchText": "<search text>"
  }
}'

import requests

response = requests.post(
    "https://api.gov.nominal.io/api/extractors/v1/container/search",
    headers={"Authorization": "Bearer <api key>"},
    json={"query": {"type": "searchText", "searchText": "<search text>"}},
)
response.raise_for_status()
print(response.json())

import httpx

response = httpx.post(
    "https://api.gov.nominal.io/api/extractors/v1/container/search",
    headers={"Authorization": "Bearer <api key>"},
    json={"query": {"type": "searchText", "searchText": "<search text>"}},
)
response.raise_for_status()
print(response.json())

import asyncio

import aiohttp


async def main() -> None:
    async with aiohttp.ClientSession() as session:
        async with session.post(
            "https://api.gov.nominal.io/api/extractors/v1/container/search",
            headers={"Authorization": "Bearer <api key>"},
            json={"query": {"type": "searchText", "searchText": "<search text>"}},
        ) as response:
            response.raise_for_status()
            print(await response.json())


asyncio.run(main())

Response: 200

[
  {
    "rid": "<containerized extractor rid>",
    "name": "<name>",
    "description": "<description>",
    "image": {
      "registry": "<registry>",
      "repository": "<repository>",
      "tagDetails": {
        "tags": [
          "<tags>"
        ],
        "defaultTag": "<default tag>"
      },
      "authentication": {
        "type": "userAndPassword",
        "userAndPassword": {
          "username": "<username>",
          "passwordSecretRid": "<secret rid>"
        }
      },
      "command": "<command>"
    },
    "containerImageRid": "<container image rid>",
    "inputs": [
      {
        "environmentVariable": "<environment variable>",
        "name": "<name>",
        "description": "<description>",
        "fileFilters": [
          {
            "type": "suffix",
            "suffix": "<file suffix>"
          }
        ],
        "required": true
      }
    ],
    "parameters": [
      {
        "environmentVariable": "<environment variable>",
        "name": "<name>",
        "description": "<description>",
        "required": true
      }
    ],
    "properties": {
      "<key>": "<property value>"
    },
    "labels": [
      "<label>"
    ],
    "createdAt": "2026-01-01T00:00:00Z",
    "isArchived": true,
    "timestampMetadata": {
      "seriesName": "<series name>",
      "timestampType": {
        "type": "relative",
        "relative": {
          "timeUnit": "DAYS",
          "offset": "2026-01-01T00:00:00Z"
        }
      }
    },
    "outputFileFormat": "PARQUET",
    "exitCodeMappings": [
      {
        "exitCode": 1,
        "code": "<code>",
        "message": "<message>",
        "retryable": true
      }
    ],
    "resources": {
      "cpuCores": 1,
      "memoryGib": 1,
      "diskGib": 1
    }
  }
]

Archive containerized extractor#

PUT /extractors/v1/container/{extractorRid}/archive

Archive a containerized extractor.

Path parameters

extractorRid string (RID) required

​

Response

No content (HTTP 204).

Request

curl -X PUT "https://api.gov.nominal.io/api/extractors/v1/container/<extractor rid>/archive" \
  -H "Authorization: Bearer <api key>"

import requests

response = requests.put(
    "https://api.gov.nominal.io/api/extractors/v1/container/<extractor rid>/archive",
    headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()

import httpx

response = httpx.put(
    "https://api.gov.nominal.io/api/extractors/v1/container/<extractor rid>/archive",
    headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()

import asyncio

import aiohttp


async def main() -> None:
    async with aiohttp.ClientSession() as session:
        async with session.put(
            "https://api.gov.nominal.io/api/extractors/v1/container/<extractor rid>/archive",
            headers={"Authorization": "Bearer <api key>"},
        ) as response:
            response.raise_for_status()


asyncio.run(main())

Response: 204

No content.

Unarchive containerized extractor#

PUT /extractors/v1/container/{extractorRid}/unarchive

Unarchive a containerized extractor.

Path parameters

extractorRid string (RID) required

​

Response

No content (HTTP 204).

Request

curl -X PUT "https://api.gov.nominal.io/api/extractors/v1/container/<extractor rid>/unarchive" \
  -H "Authorization: Bearer <api key>"

import requests

response = requests.put(
    "https://api.gov.nominal.io/api/extractors/v1/container/<extractor rid>/unarchive",
    headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()

import httpx

response = httpx.put(
    "https://api.gov.nominal.io/api/extractors/v1/container/<extractor rid>/unarchive",
    headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()

import asyncio

import aiohttp


async def main() -> None:
    async with aiohttp.ClientSession() as session:
        async with session.put(
            "https://api.gov.nominal.io/api/extractors/v1/container/<extractor rid>/unarchive",
            headers={"Authorization": "Bearer <api key>"},
        ) as response:
            response.raise_for_status()


asyncio.run(main())

Response: 204

No content.

Errors#

Errors shows how these errors are sent.

Each error’s errorName is ContainerizedExtractor: followed by its name below.

Status

Name

Notes

400

InvalidContainerImage

400

InvalidDefaultTag

400

InvalidOutputFileFormat

404

ContainerizedExtractorNotFound

404

ContainerizedExtractorsNotFound

Types#

AbsoluteTimestamp#

Set type to the name of one of the other fields, and set only that field.

iso8601 Iso8601Timestamp

​

epochOfTimeUnit EpochTimestamp

​

customFormat CustomTimestamp

​

Authentication#

Authentication methods for Docker registries.

Set type to the name of one of the other fields, and set only that field.

userAndPassword UserAndPasswordAuthentication

​

public PublicAuthentication

​

ContainerizedExtractor#

Represents a containerized extractor that processes input files using a container.

rid string (RID) required

Unique resource identifier for the extractor.

name string required

The name of the extractor as defined by the user.

description string

Optional description of the extractor.

image DockerImageSource required deprecated

Deprecated: use containerImageRid instead for self-hosted images. Container image used to run the extractor via an external registry. When containerImageRid is set, this field contains placeholder values and should be ignored. Deprecated: Use containerImageRid for self-hosted images.

containerImageRid string (RID)

RID of a self-hosted container image in the internal registry. When set, the extractor uses the internal registry; when absent, the legacy image field is used.

inputs array of FileExtractionInput

The input files that this extractor requires, mapped to environment variables that store the path to the file.

parameters array of FileExtractionParameter

Describes the parameters of the extractor.

properties map of PropertyName to string

Additional properties associated with this extractor.

labels array of string

Set of labels applied to this extractor.

createdAt string (date-time) required

Timestamp when this extractor was created.

isArchived boolean required

Whether this extractor is archived.

timestampMetadata TimestampMetadata

Metadata about the intermediate parquet this extractor will produce. If not set, timestamp metadata must be provided at ingest time.

outputFileFormat FileOutputFormat required

The format of the output file. Currently only “parquet”, “csv”, “parquet.tar” are supported

exitCodeMappings array of ExitCodeMapping

Fallback exit-code-to-error mappings for extractors that cannot emit a structured termination-log document. Populated from the active container image for v2 extractors; empty otherwise.

resources ContainerResources

Compute resources declared on the extractor’s container image. Read-only: set at image build time and never accepted on register or update. Absent, and absent per field, for an image that declares none, in which case the deployment-wide default configured on combined-service applies.

ContainerResources#

Compute resources granted to an extractor container. Every field is optional; an unset field falls back to the deployment-wide default configured on combined-service.

cpuCores integer

CPU cores requested for the extractor container.

memoryGib integer

Memory in GiB, applied as both the request and the limit.

diskGib integer

Size in GiB of each of the ephemeral input and output volumes.

CustomTimestamp#

format string required

The format string should be in the format of the DateTimeFormatter class in Java.

defaultYear integer

Default year is accepted as an optional field for cases like IRIG time format and will be overridden by year in time format.

defaultDayOfYear integer

Default day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.

DockerImageSource#

Docker container image source definition.

registry string required

The container registry where the image is hosted.

repository string required

The repository name of the image.

tagDetails TagDetails required

​

authentication Authentication required

Optional authentication for accessing private container registries.

command string

​

EpochTimestamp#

timeUnit TimeUnit required

​

ExitCodeMapping#

Maps a container exit code to a structured error, for extractors that cannot emit a structured /dev/termination-log document. Applied as a fallback error tier at ingest time.

exitCode integer required

The container process exit code this mapping applies to.

code string required

Machine-readable error code. Extractor-defined codes must fall outside the reserved platform namespace.

message string required

Human-readable error message.

retryable boolean required

Whether the platform may retry the ingest for this failure.

FileExtractionInput#

Defines an input file to be provided to the extractor.

environmentVariable string required

The environment variable that stores the path to the input file.

name string required

Name of the input file which users will be prompted with

description string

Description of the input file which users will be prompted with

fileFilters array of FileFilter

Optionally filter files for file selection

required boolean

Whether the input file is required for the extractor to run.

FileExtractionParameter#

Defines an input parameter to be provided to the extractor.

environmentVariable string required

The environment variable that stores the argument

name string required

Name of the parameter which users will be prompted with

description string

Description of the parameter which users will be prompted with

required boolean

Whether the parameter is required for the extractor to run.

FileFilter#

Set type to the name of one of the other fields, and set only that field.

suffix string

​

FileOutputFormat#

PARQUET

​

CSV

​

PARQUET_TAR

​

AVRO_STREAM

​

JSON_L

​

MANIFEST

​

Iso8601Timestamp#

An empty JSON object, {}.

Property#

name string required

​

value string required

​

PublicAuthentication#

An empty JSON object, {}.

RelativeTimestamp#

timeUnit TimeUnit required

​

offset string (date-time)

Starting timestamp to use when indexing the file. This field is required when uploading a new file to an existing dataset.

SearchContainerizedExtractorsQuery#

Set type to the name of one of the other fields, and set only that field.

searchText string

​

label string

​

property Property

​

and array of SearchContainerizedExtractorsQuery

​

or array of SearchContainerizedExtractorsQuery

​

workspace string (RID)

​

TagDetails#

tags array of string

​

defaultTag string required

​

TimestampMetadata#

seriesName string required

​

timestampType TimestampType required

​

TimestampType#

Set type to the name of one of the other fields, and set only that field.

relative RelativeTimestamp

​

absolute AbsoluteTimestamp

​

TimeUnit#

DAYS

​

HOURS

​

MINUTES

​

SECONDS

​

MILLISECONDS

​

MICROSECONDS

​

NANOSECONDS

​

UserAndPasswordAuthentication#

Username and password authentication.

username string required

Username for registry authentication.

passwordSecretRid string (RID) required

The RID of the secret containing the password for registry authentication.

Page design based on Scalar, MIT License.