Register an image#

An extractor is created first, then given images over time. Each image is a docker save tarball built against the container contract, registered with its execution contract: its inputs, parameters, output format, and default timestamp metadata. You can do this from the Web UI, the Python client, the nom CLI, or the REST API.

A few rules hold however you register:

  • Registering does not activate. Activation decides which image runs, and only an image whose push has finished (READY) can be activated. Future ingests use the newly active image, while in-flight jobs finish on the one they started with, so a rollback is reactivating the prior image and does not repair data already ingested.

  • Tags are immutable versions under the stable extractor, so prefer an existing semver or CI release identifier over a movable alias like latest. A tag must be unique among the extractor’s images. A failed registration keeps its tag, so retry with a new one.

  • A default timestamp column and type are required, even if every ingest overrides them. See Timestamps.

  • Input and parameter environment variables are how your code finds things. They must be distinct across both lists, and they are what callers pass as sources and arguments when they run an ingest.

Web UI#

  1. Navigate to the File extractors page in the Nominal Web UI.

  2. Click NEW EXTRACTOR, choose Custom extractor, and give it a name and description.

  3. In the Register image dialog that follows:

    • Image tarball: upload a docker save tarball of your image (must be built for linux/amd64).

    • Tag: identifies this image; must be unique among the extractor’s images (for example v1, v2).

    • File inputs: the environment variables your container reads to find its input files, with optional file type filters.

    • Parameters: optional string arguments passed to your container as environment variables.

    • Output format: the format of the file your container writes.

    • Timestamp Metadata: how rows in the output are indexed in time; users can override this per upload.

  4. Click REGISTER IMAGE. The image is pushed to the internal registry and becomes the extractor’s active image once the push completes.

To register a new image version for an existing extractor, select the extractor row on the File extractors page and choose Register image from the row menu.

Python#

From the extractor’s declarations#

An extractor written with the Python package already describes what it accepts. Reuse that contract when registering the image instead of maintaining a second list of inputs and parameters. If you rename an input or make a parameter optional, the generated registration metadata follows the declaration.

There are three steps, all from nominal.core: create the extractor, register an image using **split.registration_kwargs(), then activate that image.

from nominal.core import NominalClient
from extractor import split  # The decorated entrypoint saved as extractor.py.

client = NominalClient.from_profile("default")

extractor = client.create_containerized_extractor(
    "flight-recorder",
    description="Splits .flight recordings into telemetry and camera channels",
)

image = extractor.register_image(
    "my-extractor-v1.tar",
    tag="v1",
    **split.registration_kwargs(),
)

extractor.set_active_image(image)

register_image() uploads the docker save tarball and registers it. set_active_image() waits for the image to report READY before activating it; pass poll_until_ready=False to raise instead of waiting.

Notes worth knowing before you hit them:

  • registration_kwargs() is pure. It reads declarations without reading the environment or running extraction. It emits the client’s input and parameter objects and output and timestamp settings.

  • output_format is automatic for manifest extractors. For single-file metadata generation, declare @single_file_extractor(output_format=FileOutputFormat.CSV, ...) (or PARQUET / AVRO_STREAM). An explicit format is also checked against the registered format at runtime.

  • Re-registering an existing tag raises NominalAlreadyExistsError. Inspect the existing image with extractor.search_container_images(tag=...) rather than retrying under a new suffix.

  • default_timestamp_column and default_timestamp_type are required. Declare both on the extractor decorator to generate registration metadata. These settings describe image registration defaults; they do not change local output timestamp behavior.

What the generated metadata contains#

Declaration

register_image argument

@input

inputs: FileExtractionInput objects with name, environment variable, description, suffixes, and requiredness

@parameter

parameters: FileExtractionParameter objects with name, environment variable, description, and requiredness

@error

exit_code_mappings: ExitCodeMapping objects with exit code, canonical code, static message, and retryability

Outer extractor decorator

output_format, default_timestamp_column, and default_timestamp_type

Only declarations contribute to this dictionary. Generating it does not inspect the callback body, run converters, or resolve paths. It returns fresh lists on each call. Both timestamp defaults are required for generation; omitting them is allowed for runtime-only declarations. Single-file extractors also require an explicit output format for generation. The helper raises ValueError when these registration settings are incomplete.

type= and default= remain runtime settings; the current registration schema does not store them. Error mappings supply both runtime reporting and registered exit-code fallbacks. The helper does not build or upload an image, choose its tag, create an extractor, or activate it. Keep those deployment decisions in registration code, as above.

With an explicit contract#

For an image not written with the Python package, pass the contract to register_image() yourself:

from nominal.core import NominalClient
from nominal.core.container_image import FileExtractionInput, FileExtractionParameter

client = NominalClient.from_profile("default")

# An extractor carries only identity. How it runs lives on its images.
extractor = client.create_containerized_extractor(
    name="bespoke-file-parser",
    description="Parses bespoke binary logs into parquet",
)

# Uploads the `docker save` tarball and registers it as an image in one call.
# Build for linux/amd64: registration does not check the architecture, so an
# arm64 tarball registers and activates, then fails at ingest.
image = extractor.register_image(
    "bespoke-file-parser.tar",
    tag="v1",
    inputs=[
        FileExtractionInput(
            name="Input file",
            # Your container reads the file path from this environment variable
            environment_variable="INPUT_FILE",
            file_suffixes=["bin"],
            required=True,
        ),
    ],
    parameters=[
        FileExtractionParameter(
            name="Parsing mode",
            environment_variable="MODE",
            required=False,
        ),
    ],
    # How rows in the output are indexed in time. Ingests can override this.
    default_timestamp_column="timestamp",
    default_timestamp_type="iso_8601",
)

# Waits for the image to finish pushing (PENDING -> READY), then activates it.
# Activation is atomic, so this is also how you ship a new version.
extractor.set_active_image(image)

print(f"Extractor {extractor.rid} now runs image {image.rid}")

The inputs and parameters you declare here are exactly the environment variables your code reads, so they have to match.

Warning

register_image() defaults output_format to PARQUET, so an image that writes a manifest needs output_format=FileOutputFormat.MANIFEST passed explicitly. Forgetting it registers cleanly and fails at startup on the first ingest.

CLI#

The same flow is available from the nom command line that comes with the Python client, which is usually what you want in CI. The image’s execution contract lives in a checked-in JSON config, and register-image prints the new image RID on stdout so it can be captured:

EXTRACTOR_RID=$(nom container extractor create -n bespoke-file-parser -d "Parses bespoke binary logs")

IMAGE_RID=$(nom container extractor register-image -r "$EXTRACTOR_RID" \
  -f bespoke-file-parser.tar -t "$(git rev-parse --short HEAD)" -c extractor-config.json)

nom container extractor set-active-image -r "$EXTRACTOR_RID" -i "$IMAGE_RID"

The config holds inputs and parameters, as lists of objects with the snake_case fields of FileExtractionInput and FileExtractionParameter, and optionally tag, default_timestamp_column, default_timestamp_type, and output_format. Flags on the command line override fields in the file. For the image registered in the Python example above:

{
  "inputs": [
    {"name": "Input file", "environment_variable": "INPUT_FILE", "file_suffixes": ["bin"], "required": true}
  ],
  "parameters": [
    {"name": "Parsing mode", "environment_variable": "MODE", "required": false}
  ],
  "default_timestamp_column": "timestamp",
  "default_timestamp_type": "iso_8601",
  "output_format": "parquet"
}

nom container extractor validate-config extractor-config.json runs the same validation that register-image performs, without contacting Nominal or needing credentials, so it can lint the checked-in config in CI.

REST API#

Five calls, for clients that cannot use Python or the CLI. The examples use ${API_SERVER} for your Nominal API host and <token> for your API key; see API keys setup for where to find them.

1. Create the extractor#

curl -X POST ${API_SERVER}/api/extractors/v2 -H "Authorization: Bearer <token>" -H "Content-Type: application/json" -d '{
  "workspaceRid": "your_workspace_rid",
  "name": "bespoke-file-parser",
  "description": "Parses bespoke binary logs into parquet"
}'

The response contains the extractor and its RID. Save the RID; image registration and ingest requests reference it.

2. Upload the image tarball#

Save your image for linux/amd64 and upload it through the REST API’s file upload endpoint:

docker save --platform linux/amd64 bespoke-file-parser:latest -o bespoke-file-parser.tar

curl -X POST "${API_SERVER}/api/upload/v1/upload-file?fileName=bespoke-file-parser.tar&sizeBytes=$(stat -f%z bespoke-file-parser.tar)&workspace=your_workspace_rid" \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/octet-stream" \
  --data-binary @bespoke-file-parser.tar

The response is the object storage path of the uploaded tarball (for example "s3://bucket/key.tar"). Pass it as objectPath in the next step.

3. Register the image#

Register the tarball as a container image for your extractor. This triggers the push into Nominal’s internal registry:

curl -X POST ${API_SERVER}/api/twirp/nominal.registry.v2.RegistryService/CreateImage -H "Authorization: Bearer <token>" -H "Content-Type: application/json" -d '{
  "workspaceRid": "your_workspace_rid",
  "extractorRid": "your_extractor_rid",
  "tag": "v1",
  "objectPath": "s3://bucket/key.tar",
  "inputs": [
    {
      "environmentVariable": "INPUT_FILE",
      "name": "Input file",
      "fileFilters": [{ "suffix": { "suffix": "bin" } }],
      "required": true
    }
  ],
  "parameters": [
    {
      "environmentVariable": "MODE",
      "name": "Parsing mode",
      "required": false
    }
  ],
  "fileOutputFormat": "FILE_OUTPUT_FORMAT_PARQUET",
  "defaultTimestampMetadata": {
    "seriesName": "timestamp",
    "timestampType": {
      "absolute": { "iso8601": {} }
    }
  }
}'

Request fields

  • tag: identifies this image; must be unique among the extractor’s images. A failed registration keeps its tag, so retry with a new one.

  • objectPath: the path returned by the upload in step 2.

  • inputs: the input files your container reads, keyed by environment variable. At least one input is required. fileFilters optionally restricts file selection by suffix.

  • parameters: optional string arguments passed to the container as environment variables.

  • fileOutputFormat: one of FILE_OUTPUT_FORMAT_MANIFEST, FILE_OUTPUT_FORMAT_PARQUET, FILE_OUTPUT_FORMAT_CSV, or FILE_OUTPUT_FORMAT_AVRO_STREAM. Use FILE_OUTPUT_FORMAT_MANIFEST when your container writes multiple output files. Other values on the enum cannot be ingested, so an image registered with one produces an extractor whose every ingest fails.

  • defaultTimestampMetadata (required): how rows in the output are indexed in time. Ingest requests can override it. timestampType is one of:

    { "absolute": { "iso8601": {} } }
{ "absolute": { "epochOfTimeUnit": { "timeUnit": "MILLISECONDS" } } }
{ "absolute": { "customFormat": { "format": "yyyy-MM-dd HH:mm:ss", "defaultYear": 2026 } } }
{ "relative": { "timeUnit": "SECONDS" } }

4. Wait for the push to complete#

The response from step 3 contains the image RID and a status of CONTAINER_IMAGE_STATUS_PENDING. Poll until it reaches CONTAINER_IMAGE_STATUS_READY:

curl -X POST ${API_SERVER}/api/twirp/nominal.registry.v2.RegistryService/GetImage -H "Authorization: Bearer <token>" -H "Content-Type: application/json" -d '{
  "rid": "your_image_rid",
  "workspaceRid": "your_workspace_rid"
}'

A status of CONTAINER_IMAGE_STATUS_FAILED usually means the tarball was not a docker save archive or was not built for linux/amd64.

5. Activate the image#

Point the extractor at the new image. Only READY images built for this extractor can be activated:

curl -X PATCH ${API_SERVER}/api/extractors/v2/your_extractor_rid -H "Authorization: Bearer <token>" -H "Content-Type: application/json" -d '{
  "workspaceRid": "your_workspace_rid",
  "activeContainerImageRid": "your_image_rid"
}'

The extractor now appears in the upload flow with the active image’s inputs and parameters, and is ready to ingest. To ship a new version later, repeat steps 2 through 5 with a new tag; activating the new image is an atomic switch.