Containerized extractor service (gRPC)#

Manages containerized extractors backed by self-hosted images in the internal registry.

Endpoints#

Archive containerized extractor#

gRPC /nominal.ingest.v2.ContainerizedExtractorService/ArchiveContainerizedExtractor

Archive an extractor so it is hidden from default search results and rejects new ingests.

Request: ArchiveContainerizedExtractorRequest

rid string required
workspace_rid string required

Response: ArchiveContainerizedExtractorResponse

A message with no fields.

Request

Python

import grpc
from nominal.protos.ingest.v2 import containerized_extractor_pb2, containerized_extractor_pb2_grpc

channel = grpc.secure_channel("api.gov.nominal.io:443", grpc.ssl_channel_credentials())
containerized_extractor_service = containerized_extractor_pb2_grpc.ContainerizedExtractorServiceStub(channel)
auth = [("authorization", "Bearer <api key>")]

request = containerized_extractor_pb2.ArchiveContainerizedExtractorRequest(
    rid="<rid>",
    workspace_rid="<workspace rid>",
)
response = containerized_extractor_service.ArchiveContainerizedExtractor(request, metadata=auth)
print(response)

package main

import (
	"context"
	"crypto/tls"
	"fmt"
	"log"

	ingestv2 "github.com/nominal-io/nominal-api-protos-go/nominal/protos/ingest/v2"
	"google.golang.org/grpc"
	"google.golang.org/grpc/credentials"
	"google.golang.org/grpc/metadata"
)

func main() {
	conn, err := grpc.NewClient("api.gov.nominal.io:443",
		grpc.WithTransportCredentials(credentials.NewTLS(&tls.Config{})))
	if err != nil {
		log.Fatal(err)
	}
	defer conn.Close()
	containerizedExtractorService := ingestv2.NewContainerizedExtractorServiceClient(conn)
	ctx := metadata.AppendToOutgoingContext(context.Background(), "authorization", "Bearer <api key>")

	request := &ingestv2.ArchiveContainerizedExtractorRequest{
		Rid:          "<rid>",
		WorkspaceRid: "<workspace rid>",
	}
	response, err := containerizedExtractorService.ArchiveContainerizedExtractor(ctx, request)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(response)
}

Response

{}

Create containerized extractor#

gRPC /nominal.ingest.v2.ContainerizedExtractorService/CreateContainerizedExtractor

Create a new extractor in a workspace. Container images are attached separately via RegistryService.CreateImage.

Request: CreateContainerizedExtractorRequest

workspace_rid string required
name string required
description string
properties map of string to string: Free-form name/value pairs to associate with the new extractor. Uses the resource metadata limits above. At most 64 entries.
labels array of string: Labels to apply to the new extractor. Uses the resource metadata limits above. At most 64 items. Each item: 1 to 255 characters.

Response: CreateContainerizedExtractorResponse

extractor ContainerizedExtractor

Fields of ContainerizedExtractor

rid string

Unique resource identifier for this extractor.

workspace_rid string

Workspace that owns this extractor.

name string

User-supplied name. Need not be unique within a workspace.

description string

Free-form description shown in the UI.

created_at string (date-time)

Timestamp at which this extractor was first registered.

is_archived boolean

True once the extractor has been archived. Archived extractors are hidden from default search results and reject new ingests; flip back to false on UpdateContainerizedExtractor to restore.

active_container_image ContainerImage

The container image currently selected to run this extractor. Unset until the first image for this extractor reaches READY status; ingest requests that arrive before then will fail. The schema fields (inputs, parameters, output_file_format, default_timestamp_metadata) live on this nested image.

Fields of ContainerImage

rid string

tag string

User-supplied tag, unique within (workspace, extractor).

size_bytes integer (int64)

Size of the stored image in bytes once the image reaches READY.

status ContainerImageStatus

Current lifecycle status (PENDING while pushing, READY after a successful push, FAILED otherwise).

Values of ContainerImageStatus

CONTAINER_IMAGE_STATUS_UNSPECIFIED
CONTAINER_IMAGE_STATUS_PENDING
CONTAINER_IMAGE_STATUS_READY
CONTAINER_IMAGE_STATUS_FAILED

created_at string (date-time)

When the image record was created.

extractor_rid string

The containerized extractor this image is built for.

inputs array of FileExtractionInput

The input files this extractor requires, mapped to environment variables that store the path to each file.

Fields of FileExtractionInput

environment_variable string

The environment variable that stores the path to the input file.

name string

Name of the input file which users will be prompted with.

description string

Description of the input file which users will be prompted with.

file_filters array of FileFilter

Optionally filter files for file selection.

Fields of FileFilter

suffix FileSuffix

Fields of FileSuffix

suffix string

required boolean

Whether the input file is required for the extractor to run.

parameters array of FileExtractionParameter

Parameters the extractor accepts.

Fields of FileExtractionParameter

environment_variable string: The environment variable that stores the argument.
name string: Name of the parameter which users will be prompted with.
description string: Description of the parameter which users will be prompted with.
required boolean: Whether the parameter is required for the extractor to run.

file_output_format FileOutputFormat

Format of the output file produced by the extractor.

Values of FileOutputFormat

FILE_OUTPUT_FORMAT_UNSPECIFIED
FILE_OUTPUT_FORMAT_PARQUET
FILE_OUTPUT_FORMAT_CSV
FILE_OUTPUT_FORMAT_PARQUET_TAR
FILE_OUTPUT_FORMAT_AVRO_STREAM
FILE_OUTPUT_FORMAT_JSON_L
FILE_OUTPUT_FORMAT_MANIFEST

default_timestamp_metadata registry.v2.TimestampMetadata

Metadata about the intermediate parquet this extractor will produce. If not set, timestamp metadata must be provided at ingest time.

Fields of registry.v2.TimestampMetadata

series_name string

Name of the series within the output whose values are used as the row timestamp.

timestamp_type TimestampType

How the values in series_name should be interpreted as timestamps.

Fields of TimestampType

one of relative, absolute

relative

relative RelativeTimestamp

Fields of RelativeTimestamp

time_unit string: Unit in which relative offsets are expressed. Aliases the classic REST TimeUnit alias.
offset string (date-time): Starting timestamp to use when indexing the file.

absolute

absolute AbsoluteTimestamp

Fields of AbsoluteTimestamp

one of iso8601, epoch_of_time_unit, custom_format

iso8601

iso8601 Iso8601Timestamp

epoch_of_time_unit

epoch_of_time_unit EpochTimestamp

Fields of EpochTimestamp

time_unit string: Unit in which epoch values are expressed (e.g. SECONDS, MILLISECONDS).

custom_format

custom_format CustomTimestamp

Fields of CustomTimestamp

format string: The format string should be in the format of the DateTimeFormatter class in Java.
default_year integer: Default year is accepted as an optional field for cases like IRIG time format. Will be overridden by year in time format.
default_day_of_year integer: Default day of year is accepted as an optional field for cases like IRIG time format. Will be overridden by day of year in time format. From 1 to 366.

exit_code_mappings array of ExitCodeMapping

Fallback exit-code-to-error mappings for extractors that cannot emit a structured /dev/termination-log document.

Fields of ExitCodeMapping

exit_code integer: The container process exit code this mapping applies to.
code string: Machine-readable error code. Extractor-defined codes must fall outside the reserved platform namespace.
message string: Human-readable error message.
retryable boolean: Whether the platform may retry the ingest for this failure.

resources ContainerResources

Compute resources this image’s extractor container runs with. Unset fields fall back to the deployment-wide defaults.

Fields of ContainerResources

cpu_cores integer (int64): CPU cores requested for the container. From 1 to 32.
memory_gib integer (int64): Memory in GiB, applied as both the request and the limit. From 1 to 128.
disk_gib integer (int64): Size in GiB of each of the ephemeral input and output volumes. From 1 to 512.

origin ExtractorOrigin

Provenance: CUSTOMER for workspace-authored extractors (the default), FIRST_PARTY for Nominal-published extractors installed from the catalog. Read-only: native-extractor rows are created and upgraded by catalog reconciliation, never through the public write APIs.

Values of ExtractorOrigin

EXTRACTOR_ORIGIN_UNSPECIFIED
EXTRACTOR_ORIGIN_CUSTOMER
EXTRACTOR_ORIGIN_FIRST_PARTY

catalog_id string

Stable catalog identifier (e.g. “ardupilot-dataflash”). Set only when origin is FIRST_PARTY. The version currently active is carried by the installation record and by the active container image’s tag, not duplicated here.

properties map of string to string

Free-form name/value pairs associated with this extractor. At most 64 pairs; names are 1-255 chars and values are up to 1024 chars. At most 64 entries.

labels array of string

Labels applied to this extractor, filterable through SearchContainerizedExtractors. At most 64 labels of 1-255 chars each. At most 64 items. Each item: 1 to 255 characters.

Request

Python

import grpc
from nominal.protos.ingest.v2 import containerized_extractor_pb2, containerized_extractor_pb2_grpc

channel = grpc.secure_channel("api.gov.nominal.io:443", grpc.ssl_channel_credentials())
containerized_extractor_service = containerized_extractor_pb2_grpc.ContainerizedExtractorServiceStub(channel)
auth = [("authorization", "Bearer <api key>")]

request = containerized_extractor_pb2.CreateContainerizedExtractorRequest(
    workspace_rid="<workspace rid>",
    name="<name>",
)
response = containerized_extractor_service.CreateContainerizedExtractor(request, metadata=auth)
print(response)

package main

import (
	"context"
	"crypto/tls"
	"fmt"
	"log"

	ingestv2 "github.com/nominal-io/nominal-api-protos-go/nominal/protos/ingest/v2"
	"google.golang.org/grpc"
	"google.golang.org/grpc/credentials"
	"google.golang.org/grpc/metadata"
)

func main() {
	conn, err := grpc.NewClient("api.gov.nominal.io:443",
		grpc.WithTransportCredentials(credentials.NewTLS(&tls.Config{})))
	if err != nil {
		log.Fatal(err)
	}
	defer conn.Close()
	containerizedExtractorService := ingestv2.NewContainerizedExtractorServiceClient(conn)
	ctx := metadata.AppendToOutgoingContext(context.Background(), "authorization", "Bearer <api key>")

	request := &ingestv2.CreateContainerizedExtractorRequest{
		WorkspaceRid: "<workspace rid>",
		Name:         "<name>",
	}
	response, err := containerizedExtractorService.CreateContainerizedExtractor(ctx, request)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(response)
}

Response

{
  "extractor": {
    "rid": "<rid>",
    "workspace_rid": "<workspace rid>",
    "name": "<name>",
    "description": "<description>",
    "created_at": "2026-01-01T00:00:00Z",
    "is_archived": true,
    "active_container_image": {
      "rid": "<rid>",
      "tag": "<tag>",
      "size_bytes": 1,
      "status": "CONTAINER_IMAGE_STATUS_PENDING",
      "created_at": "2026-01-01T00:00:00Z",
      "extractor_rid": "<extractor rid>",
      "inputs": [
        {
          "environment_variable": "<environment variable>",
          "name": "<name>",
          "description": "<description>",
          "file_filters": [
            {
              "suffix": {
                "suffix": "<suffix>"
              }
            }
          ],
          "required": true
        }
      ],
      "parameters": [
        {
          "environment_variable": "<environment variable>",
          "name": "<name>",
          "description": "<description>",
          "required": true
        }
      ],
      "file_output_format": "FILE_OUTPUT_FORMAT_PARQUET",
      "default_timestamp_metadata": {
        "series_name": "<series name>",
        "timestamp_type": {
          "relative": {
            "time_unit": "<time unit>",
            "offset": "2026-01-01T00:00:00Z"
          }
        }
      },
      "exit_code_mappings": [
        {
          "exit_code": 1,
          "code": "<code>",
          "message": "<message>",
          "retryable": true
        }
      ],
      "resources": {
        "cpu_cores": 1,
        "memory_gib": 1,
        "disk_gib": 1
      }
    },
    "origin": "EXTRACTOR_ORIGIN_CUSTOMER",
    "catalog_id": "<catalog id>",
    "properties": {
      "<key>": "<value>"
    },
    "labels": [
      "<labels>"
    ]
  }
}

Get containerized extractor#

gRPC /nominal.ingest.v2.ContainerizedExtractorService/GetContainerizedExtractor

Fetch a single extractor by RID.

Request: GetContainerizedExtractorRequest

rid string required
workspace_rid string required

Response: GetContainerizedExtractorResponse

extractor ContainerizedExtractor

Fields of ContainerizedExtractor

rid string

Unique resource identifier for this extractor.

workspace_rid string

Workspace that owns this extractor.

name string

User-supplied name. Need not be unique within a workspace.

description string

Free-form description shown in the UI.

created_at string (date-time)

Timestamp at which this extractor was first registered.

is_archived boolean

True once the extractor has been archived. Archived extractors are hidden from default search results and reject new ingests; flip back to false on UpdateContainerizedExtractor to restore.

active_container_image ContainerImage

Fields of ContainerImage

rid string

tag string

User-supplied tag, unique within (workspace, extractor).

size_bytes integer (int64)

Size of the stored image in bytes once the image reaches READY.

status ContainerImageStatus

Current lifecycle status (PENDING while pushing, READY after a successful push, FAILED otherwise).

Values of ContainerImageStatus

CONTAINER_IMAGE_STATUS_UNSPECIFIED
CONTAINER_IMAGE_STATUS_PENDING
CONTAINER_IMAGE_STATUS_READY
CONTAINER_IMAGE_STATUS_FAILED

created_at string (date-time)

When the image record was created.

extractor_rid string

The containerized extractor this image is built for.

inputs array of FileExtractionInput

The input files this extractor requires, mapped to environment variables that store the path to each file.

Fields of FileExtractionInput

environment_variable string

The environment variable that stores the path to the input file.

name string

Name of the input file which users will be prompted with.

description string

Description of the input file which users will be prompted with.

file_filters array of FileFilter

Optionally filter files for file selection.

Fields of FileFilter

suffix FileSuffix

Fields of FileSuffix

suffix string

required boolean

Whether the input file is required for the extractor to run.

parameters array of FileExtractionParameter

Parameters the extractor accepts.

Fields of FileExtractionParameter

environment_variable string: The environment variable that stores the argument.
name string: Name of the parameter which users will be prompted with.
description string: Description of the parameter which users will be prompted with.
required boolean: Whether the parameter is required for the extractor to run.

file_output_format FileOutputFormat

Format of the output file produced by the extractor.

Values of FileOutputFormat

FILE_OUTPUT_FORMAT_UNSPECIFIED
FILE_OUTPUT_FORMAT_PARQUET
FILE_OUTPUT_FORMAT_CSV
FILE_OUTPUT_FORMAT_PARQUET_TAR
FILE_OUTPUT_FORMAT_AVRO_STREAM
FILE_OUTPUT_FORMAT_JSON_L
FILE_OUTPUT_FORMAT_MANIFEST

default_timestamp_metadata registry.v2.TimestampMetadata

Metadata about the intermediate parquet this extractor will produce. If not set, timestamp metadata must be provided at ingest time.

Fields of registry.v2.TimestampMetadata

series_name string

Name of the series within the output whose values are used as the row timestamp.

timestamp_type TimestampType

How the values in series_name should be interpreted as timestamps.

Fields of TimestampType

one of relative, absolute

relative

relative RelativeTimestamp

Fields of RelativeTimestamp

time_unit string: Unit in which relative offsets are expressed. Aliases the classic REST TimeUnit alias.
offset string (date-time): Starting timestamp to use when indexing the file.

absolute

absolute AbsoluteTimestamp

Fields of AbsoluteTimestamp

one of iso8601, epoch_of_time_unit, custom_format

iso8601

iso8601 Iso8601Timestamp

epoch_of_time_unit

epoch_of_time_unit EpochTimestamp

Fields of EpochTimestamp

time_unit string: Unit in which epoch values are expressed (e.g. SECONDS, MILLISECONDS).

custom_format

custom_format CustomTimestamp

Fields of CustomTimestamp

format string: The format string should be in the format of the DateTimeFormatter class in Java.
default_year integer: Default year is accepted as an optional field for cases like IRIG time format. Will be overridden by year in time format.
default_day_of_year integer: Default day of year is accepted as an optional field for cases like IRIG time format. Will be overridden by day of year in time format. From 1 to 366.

exit_code_mappings array of ExitCodeMapping

Fallback exit-code-to-error mappings for extractors that cannot emit a structured /dev/termination-log document.

Fields of ExitCodeMapping

exit_code integer: The container process exit code this mapping applies to.
code string: Machine-readable error code. Extractor-defined codes must fall outside the reserved platform namespace.
message string: Human-readable error message.
retryable boolean: Whether the platform may retry the ingest for this failure.

resources ContainerResources

Compute resources this image’s extractor container runs with. Unset fields fall back to the deployment-wide defaults.

Fields of ContainerResources

cpu_cores integer (int64): CPU cores requested for the container. From 1 to 32.
memory_gib integer (int64): Memory in GiB, applied as both the request and the limit. From 1 to 128.
disk_gib integer (int64): Size in GiB of each of the ephemeral input and output volumes. From 1 to 512.

origin ExtractorOrigin

Values of ExtractorOrigin

EXTRACTOR_ORIGIN_UNSPECIFIED
EXTRACTOR_ORIGIN_CUSTOMER
EXTRACTOR_ORIGIN_FIRST_PARTY

catalog_id string

properties map of string to string

Free-form name/value pairs associated with this extractor. At most 64 pairs; names are 1-255 chars and values are up to 1024 chars. At most 64 entries.

labels array of string

Labels applied to this extractor, filterable through SearchContainerizedExtractors. At most 64 labels of 1-255 chars each. At most 64 items. Each item: 1 to 255 characters.

Request

Python

import grpc
from nominal.protos.ingest.v2 import containerized_extractor_pb2, containerized_extractor_pb2_grpc

channel = grpc.secure_channel("api.gov.nominal.io:443", grpc.ssl_channel_credentials())
containerized_extractor_service = containerized_extractor_pb2_grpc.ContainerizedExtractorServiceStub(channel)
auth = [("authorization", "Bearer <api key>")]

request = containerized_extractor_pb2.GetContainerizedExtractorRequest(
    rid="<rid>",
    workspace_rid="<workspace rid>",
)
response = containerized_extractor_service.GetContainerizedExtractor(request, metadata=auth)
print(response)

package main

import (
	"context"
	"crypto/tls"
	"fmt"
	"log"

	ingestv2 "github.com/nominal-io/nominal-api-protos-go/nominal/protos/ingest/v2"
	"google.golang.org/grpc"
	"google.golang.org/grpc/credentials"
	"google.golang.org/grpc/metadata"
)

func main() {
	conn, err := grpc.NewClient("api.gov.nominal.io:443",
		grpc.WithTransportCredentials(credentials.NewTLS(&tls.Config{})))
	if err != nil {
		log.Fatal(err)
	}
	defer conn.Close()
	containerizedExtractorService := ingestv2.NewContainerizedExtractorServiceClient(conn)
	ctx := metadata.AppendToOutgoingContext(context.Background(), "authorization", "Bearer <api key>")

	request := &ingestv2.GetContainerizedExtractorRequest{
		Rid:          "<rid>",
		WorkspaceRid: "<workspace rid>",
	}
	response, err := containerizedExtractorService.GetContainerizedExtractor(ctx, request)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(response)
}

Response

{
  "extractor": {
    "rid": "<rid>",
    "workspace_rid": "<workspace rid>",
    "name": "<name>",
    "description": "<description>",
    "created_at": "2026-01-01T00:00:00Z",
    "is_archived": true,
    "active_container_image": {
      "rid": "<rid>",
      "tag": "<tag>",
      "size_bytes": 1,
      "status": "CONTAINER_IMAGE_STATUS_PENDING",
      "created_at": "2026-01-01T00:00:00Z",
      "extractor_rid": "<extractor rid>",
      "inputs": [
        {
          "environment_variable": "<environment variable>",
          "name": "<name>",
          "description": "<description>",
          "file_filters": [
            {
              "suffix": {
                "suffix": "<suffix>"
              }
            }
          ],
          "required": true
        }
      ],
      "parameters": [
        {
          "environment_variable": "<environment variable>",
          "name": "<name>",
          "description": "<description>",
          "required": true
        }
      ],
      "file_output_format": "FILE_OUTPUT_FORMAT_PARQUET",
      "default_timestamp_metadata": {
        "series_name": "<series name>",
        "timestamp_type": {
          "relative": {
            "time_unit": "<time unit>",
            "offset": "2026-01-01T00:00:00Z"
          }
        }
      },
      "exit_code_mappings": [
        {
          "exit_code": 1,
          "code": "<code>",
          "message": "<message>",
          "retryable": true
        }
      ],
      "resources": {
        "cpu_cores": 1,
        "memory_gib": 1,
        "disk_gib": 1
      }
    },
    "origin": "EXTRACTOR_ORIGIN_CUSTOMER",
    "catalog_id": "<catalog id>",
    "properties": {
      "<key>": "<value>"
    },
    "labels": [
      "<labels>"
    ]
  }
}

Search containerized extractors#

gRPC /nominal.ingest.v2.ContainerizedExtractorService/SearchContainerizedExtractors

Search extractors within a workspace, with cursor-based pagination.

Request: SearchContainerizedExtractorsRequest

workspace_rid string required
include_archived boolean: When true, archived extractors are also returned.
page_size integer required: From 1 to 1,000.
next_page_token string
file_extension string: When set, restricts results to extractors whose active container image is READY and declares at least one FileExtractionInput whose file_filters include this suffix. Example values: “csv”, “parquet”, “mcap” — no leading dot, case-sensitive against FileSuffix.suffix. 1 to 32 characters.
properties map of string to string: When non-empty, restricts results to extractors whose properties contain every one of these name/value pairs (exact match, AND semantics). Uses the resource metadata limits above. At most 64 entries.
labels array of string: When non-empty, restricts results to extractors carrying all of these labels (exact match, AND semantics). Uses the resource metadata limits above. At most 64 items. Each item: 1 to 255 characters.

Response: SearchContainerizedExtractorsResponse

extractors array of ContainerizedExtractor

Fields of ContainerizedExtractor

rid string

Unique resource identifier for this extractor.

workspace_rid string

Workspace that owns this extractor.

name string

User-supplied name. Need not be unique within a workspace.

description string

Free-form description shown in the UI.

created_at string (date-time)

Timestamp at which this extractor was first registered.

is_archived boolean

True once the extractor has been archived. Archived extractors are hidden from default search results and reject new ingests; flip back to false on UpdateContainerizedExtractor to restore.

active_container_image ContainerImage

Fields of ContainerImage

rid string

tag string

User-supplied tag, unique within (workspace, extractor).

size_bytes integer (int64)

Size of the stored image in bytes once the image reaches READY.

status ContainerImageStatus

Current lifecycle status (PENDING while pushing, READY after a successful push, FAILED otherwise).

Values of ContainerImageStatus

CONTAINER_IMAGE_STATUS_UNSPECIFIED
CONTAINER_IMAGE_STATUS_PENDING
CONTAINER_IMAGE_STATUS_READY
CONTAINER_IMAGE_STATUS_FAILED

created_at string (date-time)

When the image record was created.

extractor_rid string

The containerized extractor this image is built for.

inputs array of FileExtractionInput

The input files this extractor requires, mapped to environment variables that store the path to each file.

Fields of FileExtractionInput

environment_variable string

The environment variable that stores the path to the input file.

name string

Name of the input file which users will be prompted with.

description string

Description of the input file which users will be prompted with.

file_filters array of FileFilter

Optionally filter files for file selection.

Fields of FileFilter

suffix FileSuffix

Fields of FileSuffix

suffix string

required boolean

Whether the input file is required for the extractor to run.

parameters array of FileExtractionParameter

Parameters the extractor accepts.

Fields of FileExtractionParameter

environment_variable string: The environment variable that stores the argument.
name string: Name of the parameter which users will be prompted with.
description string: Description of the parameter which users will be prompted with.
required boolean: Whether the parameter is required for the extractor to run.

file_output_format FileOutputFormat

Format of the output file produced by the extractor.

Values of FileOutputFormat

FILE_OUTPUT_FORMAT_UNSPECIFIED
FILE_OUTPUT_FORMAT_PARQUET
FILE_OUTPUT_FORMAT_CSV
FILE_OUTPUT_FORMAT_PARQUET_TAR
FILE_OUTPUT_FORMAT_AVRO_STREAM
FILE_OUTPUT_FORMAT_JSON_L
FILE_OUTPUT_FORMAT_MANIFEST

default_timestamp_metadata registry.v2.TimestampMetadata

Metadata about the intermediate parquet this extractor will produce. If not set, timestamp metadata must be provided at ingest time.

Fields of registry.v2.TimestampMetadata

series_name string

Name of the series within the output whose values are used as the row timestamp.

timestamp_type TimestampType

How the values in series_name should be interpreted as timestamps.

Fields of TimestampType

one of relative, absolute

relative

relative RelativeTimestamp

Fields of RelativeTimestamp

time_unit string: Unit in which relative offsets are expressed. Aliases the classic REST TimeUnit alias.
offset string (date-time): Starting timestamp to use when indexing the file.

absolute

absolute AbsoluteTimestamp

Fields of AbsoluteTimestamp

one of iso8601, epoch_of_time_unit, custom_format

iso8601

iso8601 Iso8601Timestamp

epoch_of_time_unit

epoch_of_time_unit EpochTimestamp

Fields of EpochTimestamp

time_unit string: Unit in which epoch values are expressed (e.g. SECONDS, MILLISECONDS).

custom_format

custom_format CustomTimestamp

Fields of CustomTimestamp

format string: The format string should be in the format of the DateTimeFormatter class in Java.
default_year integer: Default year is accepted as an optional field for cases like IRIG time format. Will be overridden by year in time format.
default_day_of_year integer: Default day of year is accepted as an optional field for cases like IRIG time format. Will be overridden by day of year in time format. From 1 to 366.

exit_code_mappings array of ExitCodeMapping

Fallback exit-code-to-error mappings for extractors that cannot emit a structured /dev/termination-log document.

Fields of ExitCodeMapping

exit_code integer: The container process exit code this mapping applies to.
code string: Machine-readable error code. Extractor-defined codes must fall outside the reserved platform namespace.
message string: Human-readable error message.
retryable boolean: Whether the platform may retry the ingest for this failure.

resources ContainerResources

Compute resources this image’s extractor container runs with. Unset fields fall back to the deployment-wide defaults.

Fields of ContainerResources

cpu_cores integer (int64): CPU cores requested for the container. From 1 to 32.
memory_gib integer (int64): Memory in GiB, applied as both the request and the limit. From 1 to 128.
disk_gib integer (int64): Size in GiB of each of the ephemeral input and output volumes. From 1 to 512.

origin ExtractorOrigin

Values of ExtractorOrigin

EXTRACTOR_ORIGIN_UNSPECIFIED
EXTRACTOR_ORIGIN_CUSTOMER
EXTRACTOR_ORIGIN_FIRST_PARTY

catalog_id string

properties map of string to string

Free-form name/value pairs associated with this extractor. At most 64 pairs; names are 1-255 chars and values are up to 1024 chars. At most 64 entries.

labels array of string

Labels applied to this extractor, filterable through SearchContainerizedExtractors. At most 64 labels of 1-255 chars each. At most 64 items. Each item: 1 to 255 characters.

next_page_token string

Request

Python

import grpc
from nominal.protos.ingest.v2 import containerized_extractor_pb2, containerized_extractor_pb2_grpc

channel = grpc.secure_channel("api.gov.nominal.io:443", grpc.ssl_channel_credentials())
containerized_extractor_service = containerized_extractor_pb2_grpc.ContainerizedExtractorServiceStub(channel)
auth = [("authorization", "Bearer <api key>")]

request = containerized_extractor_pb2.SearchContainerizedExtractorsRequest(
    workspace_rid="<workspace rid>",
    page_size=1,
)
response = containerized_extractor_service.SearchContainerizedExtractors(request, metadata=auth)
print(response)

package main

import (
	"context"
	"crypto/tls"
	"fmt"
	"log"

	ingestv2 "github.com/nominal-io/nominal-api-protos-go/nominal/protos/ingest/v2"
	"google.golang.org/grpc"
	"google.golang.org/grpc/credentials"
	"google.golang.org/grpc/metadata"
)

func main() {
	conn, err := grpc.NewClient("api.gov.nominal.io:443",
		grpc.WithTransportCredentials(credentials.NewTLS(&tls.Config{})))
	if err != nil {
		log.Fatal(err)
	}
	defer conn.Close()
	containerizedExtractorService := ingestv2.NewContainerizedExtractorServiceClient(conn)
	ctx := metadata.AppendToOutgoingContext(context.Background(), "authorization", "Bearer <api key>")

	request := &ingestv2.SearchContainerizedExtractorsRequest{
		WorkspaceRid: "<workspace rid>",
		PageSize:     1,
	}
	response, err := containerizedExtractorService.SearchContainerizedExtractors(ctx, request)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(response)
}

Response

{
  "extractors": [
    {
      "rid": "<rid>",
      "workspace_rid": "<workspace rid>",
      "name": "<name>",
      "description": "<description>",
      "created_at": "2026-01-01T00:00:00Z",
      "is_archived": true,
      "active_container_image": {
        "rid": "<rid>",
        "tag": "<tag>",
        "size_bytes": 1,
        "status": "CONTAINER_IMAGE_STATUS_PENDING",
        "created_at": "2026-01-01T00:00:00Z",
        "extractor_rid": "<extractor rid>",
        "inputs": [
          {
            "environment_variable": "<environment variable>",
            "name": "<name>",
            "description": "<description>",
            "file_filters": [
              {
                "suffix": {
                  "suffix": "<suffix>"
                }
              }
            ],
            "required": true
          }
        ],
        "parameters": [
          {
            "environment_variable": "<environment variable>",
            "name": "<name>",
            "description": "<description>",
            "required": true
          }
        ],
        "file_output_format": "FILE_OUTPUT_FORMAT_PARQUET",
        "default_timestamp_metadata": {
          "series_name": "<series name>",
          "timestamp_type": {
            "relative": {
              "time_unit": "<time unit>",
              "offset": "2026-01-01T00:00:00Z"
            }
          }
        },
        "exit_code_mappings": [
          {
            "exit_code": 1,
            "code": "<code>",
            "message": "<message>",
            "retryable": true
          }
        ],
        "resources": {
          "cpu_cores": 1,
          "memory_gib": 1,
          "disk_gib": 1
        }
      },
      "origin": "EXTRACTOR_ORIGIN_CUSTOMER",
      "catalog_id": "<catalog id>",
      "properties": {
        "<key>": "<value>"
      },
      "labels": [
        "<labels>"
      ]
    }
  ],
  "next_page_token": "<next page token>"
}

Update containerized extractor#

gRPC /nominal.ingest.v2.ContainerizedExtractorService/UpdateContainerizedExtractor

Partial update of an existing extractor. Use the is_archived field to archive / unarchive.

Request: UpdateContainerizedExtractorRequest

rid string required

workspace_rid string required

name string

description string

is_archived boolean

Set to true to archive, false to unarchive. Leave unset to leave archive state unchanged.

active_container_image_rid string

RID of the container image to set as the active one for this extractor. Must reference a ContainerImage that is owned by the same workspace, was built for this extractor, and has reached READY status. Leave unset to leave the active image unchanged.

properties PropertyUpdateWrapper

Absent metadata is unchanged; present collections replace it, including empty collections. Uses the resource metadata limits above. properties must contain at most 64 pairs, with names of 1-255 characters and values up to 1024 characters.

Fields of PropertyUpdateWrapper

properties map of string to string

labels LabelUpdateWrapper

Uses the resource metadata limits above. labels must contain at most 64 entries of 1-255 characters each.

Fields of LabelUpdateWrapper

labels array of string

Response: UpdateContainerizedExtractorResponse

extractor ContainerizedExtractor

Fields of ContainerizedExtractor

rid string

Unique resource identifier for this extractor.

workspace_rid string

Workspace that owns this extractor.

name string

User-supplied name. Need not be unique within a workspace.

description string

Free-form description shown in the UI.

created_at string (date-time)

Timestamp at which this extractor was first registered.

is_archived boolean

True once the extractor has been archived. Archived extractors are hidden from default search results and reject new ingests; flip back to false on UpdateContainerizedExtractor to restore.

active_container_image ContainerImage

Fields of ContainerImage

rid string

tag string

User-supplied tag, unique within (workspace, extractor).

size_bytes integer (int64)

Size of the stored image in bytes once the image reaches READY.

status ContainerImageStatus

Current lifecycle status (PENDING while pushing, READY after a successful push, FAILED otherwise).

Values of ContainerImageStatus

CONTAINER_IMAGE_STATUS_UNSPECIFIED
CONTAINER_IMAGE_STATUS_PENDING
CONTAINER_IMAGE_STATUS_READY
CONTAINER_IMAGE_STATUS_FAILED

created_at string (date-time)

When the image record was created.

extractor_rid string

The containerized extractor this image is built for.

inputs array of FileExtractionInput

The input files this extractor requires, mapped to environment variables that store the path to each file.

Fields of FileExtractionInput

environment_variable string

The environment variable that stores the path to the input file.

name string

Name of the input file which users will be prompted with.

description string

Description of the input file which users will be prompted with.

file_filters array of FileFilter

Optionally filter files for file selection.

Fields of FileFilter

suffix FileSuffix

Fields of FileSuffix

suffix string

required boolean

Whether the input file is required for the extractor to run.

parameters array of FileExtractionParameter

Parameters the extractor accepts.

Fields of FileExtractionParameter

environment_variable string: The environment variable that stores the argument.
name string: Name of the parameter which users will be prompted with.
description string: Description of the parameter which users will be prompted with.
required boolean: Whether the parameter is required for the extractor to run.

file_output_format FileOutputFormat

Format of the output file produced by the extractor.

Values of FileOutputFormat

FILE_OUTPUT_FORMAT_UNSPECIFIED
FILE_OUTPUT_FORMAT_PARQUET
FILE_OUTPUT_FORMAT_CSV
FILE_OUTPUT_FORMAT_PARQUET_TAR
FILE_OUTPUT_FORMAT_AVRO_STREAM
FILE_OUTPUT_FORMAT_JSON_L
FILE_OUTPUT_FORMAT_MANIFEST

default_timestamp_metadata registry.v2.TimestampMetadata

Metadata about the intermediate parquet this extractor will produce. If not set, timestamp metadata must be provided at ingest time.

Fields of registry.v2.TimestampMetadata

series_name string

Name of the series within the output whose values are used as the row timestamp.

timestamp_type TimestampType

How the values in series_name should be interpreted as timestamps.

Fields of TimestampType

one of relative, absolute

relative

relative RelativeTimestamp

Fields of RelativeTimestamp

time_unit string: Unit in which relative offsets are expressed. Aliases the classic REST TimeUnit alias.
offset string (date-time): Starting timestamp to use when indexing the file.

absolute

absolute AbsoluteTimestamp

Fields of AbsoluteTimestamp

one of iso8601, epoch_of_time_unit, custom_format

iso8601

iso8601 Iso8601Timestamp

epoch_of_time_unit

epoch_of_time_unit EpochTimestamp

Fields of EpochTimestamp

time_unit string: Unit in which epoch values are expressed (e.g. SECONDS, MILLISECONDS).

custom_format

custom_format CustomTimestamp

Fields of CustomTimestamp

format string: The format string should be in the format of the DateTimeFormatter class in Java.
default_year integer: Default year is accepted as an optional field for cases like IRIG time format. Will be overridden by year in time format.
default_day_of_year integer: Default day of year is accepted as an optional field for cases like IRIG time format. Will be overridden by day of year in time format. From 1 to 366.

exit_code_mappings array of ExitCodeMapping

Fallback exit-code-to-error mappings for extractors that cannot emit a structured /dev/termination-log document.

Fields of ExitCodeMapping

exit_code integer: The container process exit code this mapping applies to.
code string: Machine-readable error code. Extractor-defined codes must fall outside the reserved platform namespace.
message string: Human-readable error message.
retryable boolean: Whether the platform may retry the ingest for this failure.

resources ContainerResources

Compute resources this image’s extractor container runs with. Unset fields fall back to the deployment-wide defaults.

Fields of ContainerResources

cpu_cores integer (int64): CPU cores requested for the container. From 1 to 32.
memory_gib integer (int64): Memory in GiB, applied as both the request and the limit. From 1 to 128.
disk_gib integer (int64): Size in GiB of each of the ephemeral input and output volumes. From 1 to 512.

origin ExtractorOrigin

Values of ExtractorOrigin

EXTRACTOR_ORIGIN_UNSPECIFIED
EXTRACTOR_ORIGIN_CUSTOMER
EXTRACTOR_ORIGIN_FIRST_PARTY

catalog_id string

properties map of string to string

Free-form name/value pairs associated with this extractor. At most 64 pairs; names are 1-255 chars and values are up to 1024 chars. At most 64 entries.

labels array of string

Labels applied to this extractor, filterable through SearchContainerizedExtractors. At most 64 labels of 1-255 chars each. At most 64 items. Each item: 1 to 255 characters.

Request

Python

import grpc
from nominal.protos.ingest.v2 import containerized_extractor_pb2, containerized_extractor_pb2_grpc

channel = grpc.secure_channel("api.gov.nominal.io:443", grpc.ssl_channel_credentials())
containerized_extractor_service = containerized_extractor_pb2_grpc.ContainerizedExtractorServiceStub(channel)
auth = [("authorization", "Bearer <api key>")]

request = containerized_extractor_pb2.UpdateContainerizedExtractorRequest(
    rid="<rid>",
    workspace_rid="<workspace rid>",
)
response = containerized_extractor_service.UpdateContainerizedExtractor(request, metadata=auth)
print(response)

package main

import (
	"context"
	"crypto/tls"
	"fmt"
	"log"

	ingestv2 "github.com/nominal-io/nominal-api-protos-go/nominal/protos/ingest/v2"
	"google.golang.org/grpc"
	"google.golang.org/grpc/credentials"
	"google.golang.org/grpc/metadata"
)

func main() {
	conn, err := grpc.NewClient("api.gov.nominal.io:443",
		grpc.WithTransportCredentials(credentials.NewTLS(&tls.Config{})))
	if err != nil {
		log.Fatal(err)
	}
	defer conn.Close()
	containerizedExtractorService := ingestv2.NewContainerizedExtractorServiceClient(conn)
	ctx := metadata.AppendToOutgoingContext(context.Background(), "authorization", "Bearer <api key>")

	request := &ingestv2.UpdateContainerizedExtractorRequest{
		Rid:          "<rid>",
		WorkspaceRid: "<workspace rid>",
	}
	response, err := containerizedExtractorService.UpdateContainerizedExtractor(ctx, request)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(response)
}

Response

{
  "extractor": {
    "rid": "<rid>",
    "workspace_rid": "<workspace rid>",
    "name": "<name>",
    "description": "<description>",
    "created_at": "2026-01-01T00:00:00Z",
    "is_archived": true,
    "active_container_image": {
      "rid": "<rid>",
      "tag": "<tag>",
      "size_bytes": 1,
      "status": "CONTAINER_IMAGE_STATUS_PENDING",
      "created_at": "2026-01-01T00:00:00Z",
      "extractor_rid": "<extractor rid>",
      "inputs": [
        {
          "environment_variable": "<environment variable>",
          "name": "<name>",
          "description": "<description>",
          "file_filters": [
            {
              "suffix": {
                "suffix": "<suffix>"
              }
            }
          ],
          "required": true
        }
      ],
      "parameters": [
        {
          "environment_variable": "<environment variable>",
          "name": "<name>",
          "description": "<description>",
          "required": true
        }
      ],
      "file_output_format": "FILE_OUTPUT_FORMAT_PARQUET",
      "default_timestamp_metadata": {
        "series_name": "<series name>",
        "timestamp_type": {
          "relative": {
            "time_unit": "<time unit>",
            "offset": "2026-01-01T00:00:00Z"
          }
        }
      },
      "exit_code_mappings": [
        {
          "exit_code": 1,
          "code": "<code>",
          "message": "<message>",
          "retryable": true
        }
      ],
      "resources": {
        "cpu_cores": 1,
        "memory_gib": 1,
        "disk_gib": 1
      }
    },
    "origin": "EXTRACTOR_ORIGIN_CUSTOMER",
    "catalog_id": "<catalog id>",
    "properties": {
      "<key>": "<value>"
    },
    "labels": [
      "<labels>"
    ]
  }
}

Types#

AbsoluteTimestamp#

Absolute timestamp parser variants.

Set exactly one of the fields below.

one of iso8601, epoch_of_time_unit, custom_format required

iso8601

iso8601 Iso8601Timestamp

epoch_of_time_unit

epoch_of_time_unit EpochTimestamp

custom_format

custom_format CustomTimestamp

ContainerImage#

Container image registered through the V2 (extractor-linked) registry flow. V2 images carry no user-supplied name — extractor identity replaces it as the stable handle.

rid string required
tag string: User-supplied tag, unique within (workspace, extractor).
size_bytes integer (int64): Size of the stored image in bytes once the image reaches READY.
status ContainerImageStatus: Current lifecycle status (PENDING while pushing, READY after a successful push, FAILED otherwise).
created_at string (date-time): When the image record was created.
extractor_rid string: The containerized extractor this image is built for.
inputs array of FileExtractionInput: The input files this extractor requires, mapped to environment variables that store the path to each file.
parameters array of FileExtractionParameter: Parameters the extractor accepts.
file_output_format FileOutputFormat: Format of the output file produced by the extractor.
default_timestamp_metadata registry.v2.TimestampMetadata: Metadata about the intermediate parquet this extractor will produce. If not set, timestamp metadata must be provided at ingest time.
exit_code_mappings array of ExitCodeMapping: Fallback exit-code-to-error mappings for extractors that cannot emit a structured /dev/termination-log document.
resources ContainerResources: Compute resources this image’s extractor container runs with. Unset fields fall back to the deployment-wide defaults.

ContainerImageStatus#

CONTAINER_IMAGE_STATUS_UNSPECIFIED
CONTAINER_IMAGE_STATUS_PENDING
CONTAINER_IMAGE_STATUS_READY
CONTAINER_IMAGE_STATUS_FAILED

ContainerizedExtractor#

A containerized extractor that processes input files using a self-hosted container image.

Schema fields (inputs, parameters, output format, timestamp metadata) live on the underlying ContainerImage rows; clients should read them off active_container_image. The extractor itself carries only identity, workspace ownership, user metadata (labels/properties), and lifecycle state.

rid string required: Unique resource identifier for this extractor.
workspace_rid string required: Workspace that owns this extractor.
name string: User-supplied name. Need not be unique within a workspace.
description string: Free-form description shown in the UI.
created_at string (date-time): Timestamp at which this extractor was first registered.
is_archived boolean: True once the extractor has been archived. Archived extractors are hidden from default search results and reject new ingests; flip back to false on UpdateContainerizedExtractor to restore.
active_container_image ContainerImage: The container image currently selected to run this extractor. Unset until the first image for this extractor reaches READY status; ingest requests that arrive before then will fail. The schema fields (inputs, parameters, output_file_format, default_timestamp_metadata) live on this nested image.
origin ExtractorOrigin: Provenance: CUSTOMER for workspace-authored extractors (the default), FIRST_PARTY for Nominal-published extractors installed from the catalog. Read-only: native-extractor rows are created and upgraded by catalog reconciliation, never through the public write APIs.
catalog_id string: Stable catalog identifier (e.g. “ardupilot-dataflash”). Set only when origin is FIRST_PARTY. The version currently active is carried by the installation record and by the active container image’s tag, not duplicated here.
properties map of string to string: Free-form name/value pairs associated with this extractor. At most 64 pairs; names are 1-255 chars and values are up to 1024 chars. At most 64 entries.
labels array of string: Labels applied to this extractor, filterable through SearchContainerizedExtractors. At most 64 labels of 1-255 chars each. At most 64 items. Each item: 1 to 255 characters.

ContainerResources#

Compute resources granted to the container this image runs. Every field is optional; an unset field falls back to the deployment-wide default configured on combined-service. The upper bounds are a platform ceiling, not a guarantee the cluster can satisfy the request — an unschedulable pod fails the ingest with EXTRACTOR_UNSCHEDULABLE. RegistryService re-checks them server-side, because the first-party catalog flow calls CreateImage in-process and never reaches protovalidate.

cpu_cores integer (int64): CPU cores requested for the container. From 1 to 32.
memory_gib integer (int64): Memory in GiB, applied as both the request and the limit. From 1 to 128.
disk_gib integer (int64): Size in GiB of each of the ephemeral input and output volumes. From 1 to 512.

CustomTimestamp#

Absolute timestamps parsed with a user-supplied format string.

format string: The format string should be in the format of the DateTimeFormatter class in Java.
default_year integer: Default year is accepted as an optional field for cases like IRIG time format. Will be overridden by year in time format.
default_day_of_year integer: Default day of year is accepted as an optional field for cases like IRIG time format. Will be overridden by day of year in time format. From 1 to 366.

EpochTimestamp#

Absolute timestamps expressed as an integer count of some time unit since the Unix epoch.

time_unit string: Unit in which epoch values are expressed (e.g. SECONDS, MILLISECONDS).

ExitCodeMapping#

Maps a container exit code to a structured error, for extractors that cannot emit a structured /dev/termination-log document. Applied as a fallback error tier at ingest time.

exit_code integer: The container process exit code this mapping applies to.
code string required: Machine-readable error code. Extractor-defined codes must fall outside the reserved platform namespace.
message string required: Human-readable error message.
retryable boolean: Whether the platform may retry the ingest for this failure.

ExtractorOrigin#

Provenance of a containerized extractor: authored by a customer workspace, or published by Nominal through the native extractor catalog and materialized into the workspace by catalog reconciliation.

EXTRACTOR_ORIGIN_UNSPECIFIED
EXTRACTOR_ORIGIN_CUSTOMER
EXTRACTOR_ORIGIN_FIRST_PARTY

FileExtractionInput#

Defines an input file to be provided to the extractor.

environment_variable string required: The environment variable that stores the path to the input file.
name string required: Name of the input file which users will be prompted with.
description string: Description of the input file which users will be prompted with.
file_filters array of FileFilter: Optionally filter files for file selection.
required boolean: Whether the input file is required for the extractor to run.

FileExtractionParameter#

Defines an input parameter to be provided to the extractor.

environment_variable string required: The environment variable that stores the argument.
name string required: Name of the parameter which users will be prompted with.
description string: Description of the parameter which users will be prompted with.
required boolean: Whether the parameter is required for the extractor to run.

FileFilter#

Optional filter applied during file selection for a file input.

suffix FileSuffix

FileOutputFormat#

Output file format emitted by the extractor. Currently PARQUET, CSV, and PARQUET_TAR are the primary supported formats.

FILE_OUTPUT_FORMAT_UNSPECIFIED
FILE_OUTPUT_FORMAT_PARQUET
FILE_OUTPUT_FORMAT_CSV
FILE_OUTPUT_FORMAT_PARQUET_TAR
FILE_OUTPUT_FORMAT_AVRO_STREAM
FILE_OUTPUT_FORMAT_JSON_L
FILE_OUTPUT_FORMAT_MANIFEST

FileSuffix#

The expected suffix of the file. For example, “parquet”, “json”, “csv”, etc.

suffix string required

Iso8601Timestamp#

Absolute timestamps serialized in ISO-8601 form. No additional parser configuration is needed.

A message with no fields.

LabelUpdateWrapper#

Wrapper classes to allow for optional fields on update requests

labels array of string

PropertyUpdateWrapper#

properties map of string to string

registry.v2.TimestampMetadata#

Describes how to derive a row timestamp from the extracted output.

series_name string required: Name of the series within the output whose values are used as the row timestamp.
timestamp_type TimestampType required: How the values in series_name should be interpreted as timestamps.

RelativeTimestamp#

Relative timestamp parser: interprets each row’s timestamp as an offset from a fixed origin.

time_unit string: Unit in which relative offsets are expressed. Aliases the classic REST TimeUnit alias.
offset string (date-time): Starting timestamp to use when indexing the file.

TimestampType#

Selects between relative-offset and absolute-timestamp parsing strategies.

Set exactly one of the fields below.

one of relative, absolute required

relative

relative RelativeTimestamp

absolute

absolute AbsoluteTimestamp

Page design based on Scalar, MIT License.