Catalog service#
The Catalog Service provides the ability to query for information about Datasets that are stored in the Nominal platform. A Dataset is the Nominal representation of data that has been uploaded to Nominal via a file, primarily CSV.
OpenAPI file: CatalogService.json
Endpoints#
Get enriched dataset#
GET /catalog/v1/datasets/{datasetUuid}
Path parameters
datasetUuidstring (UUID) required
Response
ridstring (RID)
uuidstring (UUID)
namestring
descriptionstring
displayNamestring
metadatamap of string to string deprecatedDeprecated: Deprecated in favor of properties.
handleHandledeprecatedDeprecated: Deprecated. Use DatasetFile#handle
Set
typeto"s3".s3S3Handle
Fields of
S3Handlebucketstring
keystring
ingestDatestring (date-time)Dataset creation time. The name is historical; use lastIngestedAt for when data was last added.
lastIngestedAtstring (date-time)When data was last added to this dataset, as the latest of a successful file ingest and streaming-session activity. Empty when the dataset has never received either.
ingestStatusIngestStatusdeprecatedDeprecated: Deprecated. Use lastIngestStatus.
Values of
IngestStatusIN_PROGRESS
COMPLETED
FAILED
DELETION_IN_PROGRESS
DELETED
originMetadataDatasetOriginMetadata
Fields of
DatasetOriginMetadatapathstringDeprecated in favor of FileOriginMetadata
xSeriesIsAbsolutebooleanDeprecated in favor of FileOriginMetadata
schemaDirectivePathstringDeprecated in favor of FileOriginMetadata
xSeriesColumnNamestringDeprecated in favor of FileOriginMetadata
xSeriesTimeUnitTimeUnitDeprecated in favor of FileOriginMetadata
Values of
TimeUnitDAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
timestampMetadataTimestampMetadataDeprecated in favor of FileOriginMetadata
Fields of
TimestampMetadataseriesNamestring
timestampTypecatalog.TimestampType
channelConfigChannelConfig
Fields of
ChannelConfigprefixTreeDelimiterstringIf set, will construct a prefix tree for channels of the dataset using the given delimiter.
lastIngestStatusIngestStatusV2
Set
typeto"success".successSuccessResult
Set
typeto"error".errorErrorResult
Fields of
ErrorResulterrorTypestring
messagestring
errorInstanceIdstring
Set
typeto"inProgress".inProgressInProgressResult
Fields of
InProgressResultinProgressDetailsInProgressDetails
Set
typeto"deletionInProgress".deletionInProgressDeletionInProgress
Set
typeto"deleted".deletedDeleted
Set
typeto"queued".queuedQueued
Set
typeto"parsing".parsingParsing
Set
typeto"ingesting".ingestingIngesting
Fields of
IngestingprogressnumberOptional fraction between 0 and 1 indicating progress through ingestion.
retentionPolicyRetentionPolicy
Fields of
RetentionPolicytypeRetentionPolicyType
Values of
RetentionPolicyTypeKEEP_UNTIL_EXPIRY
KEEP_FOREVER
expiryDaysinteger
sourcestring
boundsBounds
Fields of
BoundsstartTimestamp
Fields of
Timestampsecondsinteger (int64)
nanosinteger (int64)
endTimestamp
typedatasource.TimestampType
Values of
datasource.TimestampTypeRELATIVE
ABSOLUTE
timestampTypeWeakTimestampType
Values of
WeakTimestampTypeABSOLUTE
RELATIVE
PENDING
labelsarray of string
propertiesmap ofPropertyNameto string deprecatedDeprecated: use typedProperties
typedPropertiesmap ofPropertyNametoTypedPropertyValue
Set
typeto"stringValue".stringValuestring
Set
typeto"numericValue".numericValuenumber
granularityGranularity
Values of
GranularityNANOSECONDS
allowStreamingboolean
isArchivedboolean
defaultGroupByAndAggregationConfigDefaultGroupByAndAggregationConfig
Fields of
DefaultGroupByAndAggregationConfignumericAggregationNumericAggregationOperatorIf set, the default aggregation applied to merge points of a numeric series that share an exact timestamp.
Set
typeto"sum".sumSummation
Set
typeto"average".averageAverage
Set
typeto"min".minMinimum
Set
typeto"max".maxMaximum
Set
typeto"count".countCount
Set
typeto"standardDeviation".standardDeviationStandardDeviation
Set
typeto"rootMeanSquare".rootMeanSquareRootMeanSquare
Set
typeto"percentile".percentilePercentile
Set
typeto"udf".
migratedFromConnectionRidstring (RID)Connection RID whose data source was migrated into this dataset. Empty for datasets that were not created by the legacy connection-to-dataset migrations, or when the original connection row no longer exists.
externalConnectionMetadataExternalConnectionMetadata
Fields of
ExternalConnectionMetadataconnectionDetailsConnectionDetails
Set
typeto"timescale".timescaleTimescaleConnectionDetails
Set
typeto"influx".influxInflux2ConnectionDetails
Set
typeto"influx1".influx1Influx1ConnectionDetails
Set
typeto"nominal".nominalNominalConnectionDetails
Set
typeto"timestream".timestreamTimestreamConnectionDetails
Set
typeto"visualCrossing".visualCrossingVisualCrossingConnectionDetails
Set
typeto"bigQuery".bigQueryBigQueryConnectionDetails
connectionStatusConnectionStatus
Fields of
ConnectionStatusisConnectedboolean
limitsLimitsConfig
Fields of
LimitsConfigmaxQueryRangeconnection.api.Duration
derivedDefinitionDerivedDefinitionDefinition for a derived dataset. When present, this dataset is virtual and backed by the referenced provided Dataset definition instead of persisted dataset files.
Fields of
DerivedDefinitionspecapi.Dataset
Set
typeto"asset".assetAssetSingle asset. Expands to one grouping per data scope on the asset. Groupings are tagged by
assetRidanddataScopeso callers can safely compose multiple assets withcombine.
Set
typeto"run".runRunSingle run. Expands to one grouping per data source attached to the run: asset-typed sources contribute one grouping per data scope on the asset; dataset-typed sources contribute one grouping per dataset. Groupings are tagged by
runRidanddataScope; asset-typed groupings additionally carryassetRidso callers can safely compose multiple runs withcombine.
Set
typeto"saved".savedSavedDataset
Set
typeto"search".searchSearchDataset
Set
typeto"combine".combineCombinedDataset
Set
typeto"tag".tagTaggedDataset
Set
typeto"filter".filterFilteredDataset
Set
typeto"timeShift".timeShiftTimeShiftedDataset
Set
typeto"alignByAnchor".alignByAnchorAnchorAlignedDataset
Set
typeto"withSeries".withSeriesWithSeriesDataset
Set
typeto"dropSeries".dropSeriesDropSeriesDataset
Set
typeto"reference".referenceDatasetReference
Set
typeto"withDimensions".withDimensionsDimensionsDataset
commitCommit
Fields of
Commitidstring
resourceRidstring (RID)
parentCommitstring
mergeParentCommitstring
messagestring
isWorkingStatebooleanA working state commit is created via the
saveWorkingStateendpoint and is non-permanent. In the future, it may be compacted and not exist.committedBystring (RID)
committedAtstring (date-time)
channelSearchSplitTagKeysarray of stringTag keys whose distinct values channel search surfaces as separate series. Each listed key is retained on this dataset’s channel-search locators, so a channel carrying two values of that key appears once per value instead of collapsing into a single entry — the behavior a data scope has on an asset. Keys not listed here stay available for filtering and grouping but do not contribute to series cardinality. Every key must be produced by this dataset’s
derivedDefinition. Ingested tag keys cannot be split on.
Request
curl "https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset uuid>" \
-H "Authorization: Bearer <api key>"
import requests
response = requests.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset uuid>",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset uuid>",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset uuid>",
headers={"Authorization": "Bearer <api key>"},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"rid": "<rid>",
"uuid": "<uuid>",
"name": "<name>",
"description": "<description>",
"displayName": "<display name>",
"metadata": {
"<key>": "<value>"
},
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"ingestDate": "2026-01-01T00:00:00Z",
"lastIngestedAt": "2026-01-01T00:00:00Z",
"ingestStatus": "IN_PROGRESS",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": true,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS",
"offset": "2026-01-01T00:00:00Z"
}
}
},
"channelConfig": {
"prefixTreeDelimiter": "<prefix tree delimiter>"
}
},
"lastIngestStatus": {
"type": "success",
"success": {}
},
"retentionPolicy": {
"type": "KEEP_UNTIL_EXPIRY",
"expiryDays": 1
},
"source": "<source>",
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"timestampType": "ABSOLUTE",
"labels": [
"<label>"
],
"properties": {
"<key>": "<property value>"
},
"typedProperties": {
"<key>": {
"type": "stringValue",
"stringValue": "<property value>"
}
},
"granularity": "NANOSECONDS",
"allowStreaming": true,
"isArchived": true,
"defaultGroupByAndAggregationConfig": {
"numericAggregation": {
"type": "sum",
"sum": {}
}
},
"migratedFromConnectionRid": "<connection rid>",
"externalConnectionMetadata": {
"connectionDetails": {
"type": "timescale",
"timescale": {
"database": "<database>",
"host": "<host>",
"port": 1,
"username": "<username>",
"passwordSecretRid": "<secret rid>"
}
},
"connectionStatus": {
"isConnected": true
},
"limits": {
"maxQueryRange": {
"unit": "HOURS",
"count": 1
}
}
},
"derivedDefinition": {
"spec": {
"type": "asset",
"asset": {
"rid": {
"type": "literal",
"literal": "<literal>"
}
}
},
"commit": {
"id": "<commit id>",
"resourceRid": "<resource rid>",
"parentCommit": "<commit id>",
"mergeParentCommit": "<commit id>",
"message": "<message>",
"isWorkingState": true,
"committedBy": "<user rid>",
"committedAt": "2026-01-01T00:00:00Z"
}
},
"channelSearchSplitTagKeys": [
"<tag name>"
]
}
Get enriched datasets#
POST /catalog/v1/datasets/multiple
Body
datasetRidsarray of string (RID)
Response
The body is an array of EnrichedDataset.
Fields of EnrichedDataset
ridstring (RID)
uuidstring (UUID)
namestring
descriptionstring
displayNamestring
metadatamap of string to string deprecatedDeprecated: Deprecated in favor of properties.
handleHandledeprecatedDeprecated: Deprecated. Use DatasetFile#handle
Set
typeto"s3".s3S3Handle
ingestDatestring (date-time)Dataset creation time. The name is historical; use lastIngestedAt for when data was last added.
lastIngestedAtstring (date-time)When data was last added to this dataset, as the latest of a successful file ingest and streaming-session activity. Empty when the dataset has never received either.
ingestStatusIngestStatusdeprecatedDeprecated: Deprecated. Use lastIngestStatus.
Values of
IngestStatusIN_PROGRESS
COMPLETED
FAILED
DELETION_IN_PROGRESS
DELETED
originMetadataDatasetOriginMetadata
Fields of
DatasetOriginMetadatapathstringDeprecated in favor of FileOriginMetadata
xSeriesIsAbsolutebooleanDeprecated in favor of FileOriginMetadata
schemaDirectivePathstringDeprecated in favor of FileOriginMetadata
xSeriesColumnNamestringDeprecated in favor of FileOriginMetadata
xSeriesTimeUnitTimeUnitDeprecated in favor of FileOriginMetadata
timestampMetadataTimestampMetadataDeprecated in favor of FileOriginMetadata
channelConfigChannelConfig
lastIngestStatusIngestStatusV2
Set
typeto"success".successSuccessResult
Set
typeto"error".errorErrorResult
Set
typeto"inProgress".inProgressInProgressResult
Set
typeto"deletionInProgress".deletionInProgressDeletionInProgress
Set
typeto"deleted".deletedDeleted
Set
typeto"queued".queuedQueued
Set
typeto"parsing".parsingParsing
Set
typeto"ingesting".ingestingIngesting
retentionPolicyRetentionPolicy
Fields of
RetentionPolicytypeRetentionPolicyType
expiryDaysinteger
sourcestring
boundsBounds
Fields of
BoundsstartTimestamp
endTimestamp
typedatasource.TimestampType
timestampTypeWeakTimestampType
Values of
WeakTimestampTypeABSOLUTE
RELATIVE
PENDING
labelsarray of string
propertiesmap ofPropertyNameto string deprecatedDeprecated: use typedProperties
typedPropertiesmap ofPropertyNametoTypedPropertyValue
Set
typeto"stringValue".stringValuestring
Set
typeto"numericValue".numericValuenumber
granularityGranularity
Values of
GranularityNANOSECONDS
allowStreamingboolean
isArchivedboolean
defaultGroupByAndAggregationConfigDefaultGroupByAndAggregationConfig
Fields of
DefaultGroupByAndAggregationConfignumericAggregationNumericAggregationOperatorIf set, the default aggregation applied to merge points of a numeric series that share an exact timestamp.
migratedFromConnectionRidstring (RID)Connection RID whose data source was migrated into this dataset. Empty for datasets that were not created by the legacy connection-to-dataset migrations, or when the original connection row no longer exists.
externalConnectionMetadataExternalConnectionMetadata
Fields of
ExternalConnectionMetadataconnectionDetailsConnectionDetails
connectionStatusConnectionStatus
limitsLimitsConfig
derivedDefinitionDerivedDefinitionDefinition for a derived dataset. When present, this dataset is virtual and backed by the referenced provided Dataset definition instead of persisted dataset files.
Fields of
DerivedDefinitionspecapi.Dataset
commitCommit
channelSearchSplitTagKeysarray of stringTag keys whose distinct values channel search surfaces as separate series. Each listed key is retained on this dataset’s channel-search locators, so a channel carrying two values of that key appears once per value instead of collapsing into a single entry — the behavior a data scope has on an asset. Keys not listed here stay available for filtering and grouping but do not contribute to series cardinality. Every key must be produced by this dataset’s
derivedDefinition. Ingested tag keys cannot be split on.
Request
curl -X POST "https://api.gov.nominal.io/api/catalog/v1/datasets/multiple" \
-H "Authorization: Bearer <api key>" \
-H "Content-Type: application/json" \
--data '{
"datasetRids": [
"<dataset rids>"
]
}'
import requests
response = requests.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/multiple",
headers={"Authorization": "Bearer <api key>"},
json={"datasetRids": ["<dataset rids>"]},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/multiple",
headers={"Authorization": "Bearer <api key>"},
json={"datasetRids": ["<dataset rids>"]},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/multiple",
headers={"Authorization": "Bearer <api key>"},
json={"datasetRids": ["<dataset rids>"]},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
[
{
"rid": "<rid>",
"uuid": "<uuid>",
"name": "<name>",
"description": "<description>",
"displayName": "<display name>",
"metadata": {
"<key>": "<value>"
},
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"ingestDate": "2026-01-01T00:00:00Z",
"lastIngestedAt": "2026-01-01T00:00:00Z",
"ingestStatus": "IN_PROGRESS",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": true,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS",
"offset": "2026-01-01T00:00:00Z"
}
}
},
"channelConfig": {
"prefixTreeDelimiter": "<prefix tree delimiter>"
}
},
"lastIngestStatus": {
"type": "success",
"success": {}
},
"retentionPolicy": {
"type": "KEEP_UNTIL_EXPIRY",
"expiryDays": 1
},
"source": "<source>",
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"timestampType": "ABSOLUTE",
"labels": [
"<label>"
],
"properties": {
"<key>": "<property value>"
},
"typedProperties": {
"<key>": {
"type": "stringValue",
"stringValue": "<property value>"
}
},
"granularity": "NANOSECONDS",
"allowStreaming": true,
"isArchived": true,
"defaultGroupByAndAggregationConfig": {
"numericAggregation": {
"type": "sum",
"sum": {}
}
},
"migratedFromConnectionRid": "<connection rid>",
"externalConnectionMetadata": {
"connectionDetails": {
"type": "timescale",
"timescale": {
"database": "<database>",
"host": "<host>",
"port": 1,
"username": "<username>",
"passwordSecretRid": "<secret rid>"
}
},
"connectionStatus": {
"isConnected": true
},
"limits": {
"maxQueryRange": {
"unit": "HOURS",
"count": 1
}
}
},
"derivedDefinition": {
"spec": {
"type": "asset",
"asset": {
"rid": {
"type": "literal",
"literal": "<literal>"
}
}
},
"commit": {
"id": "<commit id>",
"resourceRid": "<resource rid>",
"parentCommit": "<commit id>",
"mergeParentCommit": "<commit id>",
"message": "<message>",
"isWorkingState": true,
"committedBy": "<user rid>",
"committedAt": "2026-01-01T00:00:00Z"
}
},
"channelSearchSplitTagKeys": [
"<tag name>"
]
}
]
Get dataset#
GET /catalog/v1/datasets-simple/{datasetUuid}
Path parameters
datasetUuidstring (UUID) required
Response
ridstring (RID)
namestring
handleHandledeprecatedDeprecated: Deprecated. Use DatasetFile#handle
Set
typeto"s3".s3S3Handle
Fields of
S3Handlebucketstring
keystring
descriptionstring
originMetadataDatasetOriginMetadata
Fields of
DatasetOriginMetadatapathstringDeprecated in favor of FileOriginMetadata
xSeriesIsAbsolutebooleanDeprecated in favor of FileOriginMetadata
schemaDirectivePathstringDeprecated in favor of FileOriginMetadata
xSeriesColumnNamestringDeprecated in favor of FileOriginMetadata
xSeriesTimeUnitTimeUnitDeprecated in favor of FileOriginMetadata
Values of
TimeUnitDAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
timestampMetadataTimestampMetadataDeprecated in favor of FileOriginMetadata
Fields of
TimestampMetadataseriesNamestring
timestampTypecatalog.TimestampType
channelConfigChannelConfig
Fields of
ChannelConfigprefixTreeDelimiterstringIf set, will construct a prefix tree for channels of the dataset using the given delimiter.
boundsBounds
Fields of
BoundsstartTimestamp
Fields of
Timestampsecondsinteger (int64)
nanosinteger (int64)
endTimestamp
typedatasource.TimestampType
Values of
datasource.TimestampTypeRELATIVE
ABSOLUTE
propertiesmap ofPropertyNameto string deprecatedDeprecated: use typedProperties
typedPropertiesmap ofPropertyNametoTypedPropertyValue
Set
typeto"stringValue".stringValuestring
Set
typeto"numericValue".numericValuenumber
labelsarray of string
timestampTypeWeakTimestampType
Values of
WeakTimestampTypeABSOLUTE
RELATIVE
PENDING
allowStreamingboolean
granularityGranularity
Values of
GranularityNANOSECONDS
isArchivedboolean
defaultGroupByAndAggregationConfigDefaultGroupByAndAggregationConfig
Fields of
DefaultGroupByAndAggregationConfignumericAggregationNumericAggregationOperatorIf set, the default aggregation applied to merge points of a numeric series that share an exact timestamp.
Set
typeto"sum".sumSummation
Set
typeto"average".averageAverage
Set
typeto"min".minMinimum
Set
typeto"max".maxMaximum
Set
typeto"count".countCount
Set
typeto"standardDeviation".standardDeviationStandardDeviation
Set
typeto"rootMeanSquare".rootMeanSquareRootMeanSquare
Set
typeto"percentile".percentilePercentile
Set
typeto"udf".
derivedDefinitionDerivedDefinitionDefinition for a derived dataset. When present, this dataset is virtual and backed by the referenced provided Dataset definition instead of persisted dataset files.
Fields of
DerivedDefinitionspecapi.Dataset
Set
typeto"asset".assetAssetSingle asset. Expands to one grouping per data scope on the asset. Groupings are tagged by
assetRidanddataScopeso callers can safely compose multiple assets withcombine.
Set
typeto"run".runRunSingle run. Expands to one grouping per data source attached to the run: asset-typed sources contribute one grouping per data scope on the asset; dataset-typed sources contribute one grouping per dataset. Groupings are tagged by
runRidanddataScope; asset-typed groupings additionally carryassetRidso callers can safely compose multiple runs withcombine.
Set
typeto"saved".savedSavedDataset
Set
typeto"search".searchSearchDataset
Set
typeto"combine".combineCombinedDataset
Set
typeto"tag".tagTaggedDataset
Set
typeto"filter".filterFilteredDataset
Set
typeto"timeShift".timeShiftTimeShiftedDataset
Set
typeto"alignByAnchor".alignByAnchorAnchorAlignedDataset
Set
typeto"withSeries".withSeriesWithSeriesDataset
Set
typeto"dropSeries".dropSeriesDropSeriesDataset
Set
typeto"reference".referenceDatasetReference
Set
typeto"withDimensions".withDimensionsDimensionsDataset
commitCommit
Fields of
Commitidstring
resourceRidstring (RID)
parentCommitstring
mergeParentCommitstring
messagestring
isWorkingStatebooleanA working state commit is created via the
saveWorkingStateendpoint and is non-permanent. In the future, it may be compacted and not exist.committedBystring (RID)
committedAtstring (date-time)
channelSearchSplitTagKeysarray of stringTag keys whose distinct values channel search surfaces as separate series. Each listed key is retained on this dataset’s channel-search locators, so a channel carrying two values of that key appears once per value instead of collapsing into a single entry — the behavior a data scope has on an asset. Keys not listed here stay available for filtering and grouping but do not contribute to series cardinality. Every key must be produced by this dataset’s
derivedDefinition. Ingested tag keys cannot be split on.
Request
curl "https://api.gov.nominal.io/api/catalog/v1/datasets-simple/<dataset uuid>" \
-H "Authorization: Bearer <api key>"
import requests
response = requests.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets-simple/<dataset uuid>",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets-simple/<dataset uuid>",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets-simple/<dataset uuid>",
headers={"Authorization": "Bearer <api key>"},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"rid": "<rid>",
"name": "<name>",
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"description": "<description>",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": true,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS",
"offset": "2026-01-01T00:00:00Z"
}
}
},
"channelConfig": {
"prefixTreeDelimiter": "<prefix tree delimiter>"
}
},
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"properties": {
"<key>": "<property value>"
},
"typedProperties": {
"<key>": {
"type": "stringValue",
"stringValue": "<property value>"
}
},
"labels": [
"<label>"
],
"timestampType": "ABSOLUTE",
"allowStreaming": true,
"granularity": "NANOSECONDS",
"isArchived": true,
"defaultGroupByAndAggregationConfig": {
"numericAggregation": {
"type": "sum",
"sum": {}
}
},
"derivedDefinition": {
"spec": {
"type": "asset",
"asset": {
"rid": {
"type": "literal",
"literal": "<literal>"
}
}
},
"commit": {
"id": "<commit id>",
"resourceRid": "<resource rid>",
"parentCommit": "<commit id>",
"mergeParentCommit": "<commit id>",
"message": "<message>",
"isWorkingState": true,
"committedBy": "<user rid>",
"committedAt": "2026-01-01T00:00:00Z"
}
},
"channelSearchSplitTagKeys": [
"<tag name>"
]
}
Get datasets#
POST /catalog/v1/datasets-simple/multiple
Body
datasetRidsarray of string (RID)
Response
The body is an array of catalog.Dataset.
Fields of catalog.Dataset
ridstring (RID)
namestring
handleHandledeprecatedDeprecated: Deprecated. Use DatasetFile#handle
Set
typeto"s3".s3S3Handle
descriptionstring
originMetadataDatasetOriginMetadata
Fields of
DatasetOriginMetadatapathstringDeprecated in favor of FileOriginMetadata
xSeriesIsAbsolutebooleanDeprecated in favor of FileOriginMetadata
schemaDirectivePathstringDeprecated in favor of FileOriginMetadata
xSeriesColumnNamestringDeprecated in favor of FileOriginMetadata
xSeriesTimeUnitTimeUnitDeprecated in favor of FileOriginMetadata
timestampMetadataTimestampMetadataDeprecated in favor of FileOriginMetadata
channelConfigChannelConfig
boundsBounds
Fields of
BoundsstartTimestamp
endTimestamp
typedatasource.TimestampType
propertiesmap ofPropertyNameto string deprecatedDeprecated: use typedProperties
typedPropertiesmap ofPropertyNametoTypedPropertyValue
Set
typeto"stringValue".stringValuestring
Set
typeto"numericValue".numericValuenumber
labelsarray of string
timestampTypeWeakTimestampType
Values of
WeakTimestampTypeABSOLUTE
RELATIVE
PENDING
allowStreamingboolean
granularityGranularity
Values of
GranularityNANOSECONDS
isArchivedboolean
defaultGroupByAndAggregationConfigDefaultGroupByAndAggregationConfig
Fields of
DefaultGroupByAndAggregationConfignumericAggregationNumericAggregationOperatorIf set, the default aggregation applied to merge points of a numeric series that share an exact timestamp.
derivedDefinitionDerivedDefinitionDefinition for a derived dataset. When present, this dataset is virtual and backed by the referenced provided Dataset definition instead of persisted dataset files.
Fields of
DerivedDefinitionspecapi.Dataset
commitCommit
channelSearchSplitTagKeysarray of stringTag keys whose distinct values channel search surfaces as separate series. Each listed key is retained on this dataset’s channel-search locators, so a channel carrying two values of that key appears once per value instead of collapsing into a single entry — the behavior a data scope has on an asset. Keys not listed here stay available for filtering and grouping but do not contribute to series cardinality. Every key must be produced by this dataset’s
derivedDefinition. Ingested tag keys cannot be split on.
Request
curl -X POST "https://api.gov.nominal.io/api/catalog/v1/datasets-simple/multiple" \
-H "Authorization: Bearer <api key>" \
-H "Content-Type: application/json" \
--data '{
"datasetRids": [
"<dataset rids>"
]
}'
import requests
response = requests.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets-simple/multiple",
headers={"Authorization": "Bearer <api key>"},
json={"datasetRids": ["<dataset rids>"]},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets-simple/multiple",
headers={"Authorization": "Bearer <api key>"},
json={"datasetRids": ["<dataset rids>"]},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets-simple/multiple",
headers={"Authorization": "Bearer <api key>"},
json={"datasetRids": ["<dataset rids>"]},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
[
{
"rid": "<rid>",
"name": "<name>",
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"description": "<description>",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": true,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS",
"offset": "2026-01-01T00:00:00Z"
}
}
},
"channelConfig": {
"prefixTreeDelimiter": "<prefix tree delimiter>"
}
},
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"properties": {
"<key>": "<property value>"
},
"typedProperties": {
"<key>": {
"type": "stringValue",
"stringValue": "<property value>"
}
},
"labels": [
"<label>"
],
"timestampType": "ABSOLUTE",
"allowStreaming": true,
"granularity": "NANOSECONDS",
"isArchived": true,
"defaultGroupByAndAggregationConfig": {
"numericAggregation": {
"type": "sum",
"sum": {}
}
},
"derivedDefinition": {
"spec": {
"type": "asset",
"asset": {
"rid": {
"type": "literal",
"literal": "<literal>"
}
}
},
"commit": {
"id": "<commit id>",
"resourceRid": "<resource rid>",
"parentCommit": "<commit id>",
"mergeParentCommit": "<commit id>",
"message": "<message>",
"isWorkingState": true,
"committedBy": "<user rid>",
"committedAt": "2026-01-01T00:00:00Z"
}
},
"channelSearchSplitTagKeys": [
"<tag name>"
]
}
]
Get datasets with external connection config#
POST /catalog/v1/datasets-simple/external-config
Gets the subset of dataset RIDs that currently have external connection config attached.
Body
datasetRidsarray of string (RID)
Response
The body is an array of string (RID).
Request
curl -X POST "https://api.gov.nominal.io/api/catalog/v1/datasets-simple/external-config" \
-H "Authorization: Bearer <api key>" \
-H "Content-Type: application/json" \
--data '{
"datasetRids": [
"<dataset rids>"
]
}'
import requests
response = requests.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets-simple/external-config",
headers={"Authorization": "Bearer <api key>"},
json={"datasetRids": ["<dataset rids>"]},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets-simple/external-config",
headers={"Authorization": "Bearer <api key>"},
json={"datasetRids": ["<dataset rids>"]},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets-simple/external-config",
headers={"Authorization": "Bearer <api key>"},
json={"datasetRids": ["<dataset rids>"]},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
[
"<dataset rid>"
]
Register external connection config#
POST /catalog/v1/datasets/external-config
Registers or updates external connection configuration for an existing dataset.
Body
datasetRidstring (RID) required
connectionDetailsConnectionDetailsrequired
Set
typeto"timescale".timescaleTimescaleConnectionDetails
Fields of
TimescaleConnectionDetailsdatabasestring required
hoststring required
portinteger required
usernamestring requiredThis is also the reference to the secret containing the password
passwordSecretRidstring (RID) requiredSecret Rid of password secret stored in Secrets Service.
Set
typeto"influx".influxInflux2ConnectionDetails
Fields of
Influx2ConnectionDetailshoststring required
portinteger required
headersmap of string toHeaderValueA map of header name to value
Set
typeto"plain".plainstring
Set
typeto"secretRid".secretRidstring (RID)
orgstring required
tokenSecretRidstring (RID) requiredSecret Rid of token secret stored in Secrets Service.
Set
typeto"influx1".influx1Influx1ConnectionDetails
Fields of
Influx1ConnectionDetailshoststring required
portinteger required
headersmap of string toHeaderValueA map of header name to value
credentialsPasswordCredentials
Fields of
PasswordCredentialsusernamestring required
passwordSecretRidstring (RID) requiredSecret Rid of password secret stored in Secrets Service.
Set
typeto"nominal".nominalNominalConnectionDetails
Fields of
NominalConnectionDetailsnominalDataSourceRidstring (RID) required
Set
typeto"timestream".timestreamTimestreamConnectionDetails
Fields of
TimestreamConnectionDetailsnominalRoleArnstring requiredThe role ARN owned by Nominal that has permission to assume the externalRoleArn.
nominalRoleRegionstring requiredAWS region that contains the Nominal role.
externalRegionstring requiredAWS region that contains the Timestream instance.
externalRoleArnstring requiredThe role ARN used to make queries to Timestream. This role is owned by the AWS account that owns the Timestream instance.
databasestring requiredDatabase inside of the Timestream instance to connect to.
Set
typeto"visualCrossing".visualCrossingVisualCrossingConnectionDetails
Fields of
VisualCrossingConnectionDetailsapiKeySecretRidstring (RID) requiredSecret Rid of API key stored in Secrets Service.
Set
typeto"bigQuery".bigQueryBigQueryConnectionDetails
Fields of
BigQueryConnectionDetailsregionstring requiredThe region of the BigQuery Project (e.g. “us-east1”)
projectstring requiredThe name of the BigQuery Project
datasetstring requiredThe name of the dataset within the project
tablestring requiredThe name of the table within the dataset
serviceAccountKeySecretRidstring (RID) requiredSecret Rid of service account key stored in Secrets Service.
metadatamap of string to stringMetadata information about the external connection configuration.
requiredTagNamesarray of stringAdditional tag names that are required to construct a fully qualified series.
availableTagValuesmap ofTagNameto array of stringOptional externally supplied tag values. In most cases these are indexed by scraping.
scrapingScrapingConfig
Set
typeto"influx".influxInfluxScrapingConfig
Fields of
InfluxScrapingConfigfilterarray ofInfluxScrapingFilterFilters are ANDed together
Set
typeto"not".notInfluxScrapingFilter
Set
typeto"matchBucket".matchBucketarray of stringMatch any of the specified buckets
Set
typeto"matchMeasurement".matchMeasurementarray of stringMatch any of the specified measurements
seriesArchetypeConstructionInfluxSeriesArchetypeConstructionConfigrequired
Fields of
InfluxSeriesArchetypeConstructionConfigchannelNameComponentsarray ofInfluxChannelNameComponent
separatorstring required
Set
typeto"nominal".nominalNominalScrapingConfig
Fields of
NominalScrapingConfigchannelNameComponentsarray ofNominalChannelNameComponentchannelNameComponents will be combined, together with separator, to form a fully qualified channel name.
Set
typeto"channel".channelEmpty
Set
typeto"valueOfTagWithName".valueOfTagWithNamestringThe value of the specified TagName will be used. If the tag is not present, it will be omitted.
separatorstring required
Set
typeto"timestream".timestreamTimestreamScrapingConfig
Fields of
TimestreamScrapingConfigfilterarray ofTimestreamScrapingFilterIn order for data to be picked up by the scraper, it must match all filters in this list.
Set
typeto"matchTable".matchTablearray of stringMatch any of the specified tables in the database.
channelNameComponentsarray ofTimestreamChannelNameComponentchannelNameComponents will be combined, together with separator, to form a fully qualified channel name.
separatorstring required
maxLookBackPeriodHoursintegerThe maximum time in hours to look back over series data to find unique measure and tag combinations. If not specified, the default look back period will be used.
Set
typeto"timescale".timescalePivotedTimescaleScrapingConfig
Fields of
PivotedTimescaleScrapingConfigfilterarray ofTimescaleScrapingFilterIn order for data to be picked up by the scraper, it must match all filters in this list. To exclude chunks of data, it’s possible to use a not filter.
Set
typeto"not".Set
typeto"matchTable".matchTablearray of stringMatch any of the specified tables in the database.
Set
typeto"matchSchema".matchSchemaarray of stringMatch any of the specified schemas in the database.
timeColumnstring requiredThe name of the column that holds the timestamp.
nameColumnstring requiredThe name of the column that holds the series name.
valueColumnstring requiredThe name of the column that holds the series values.
dimensionColumnsarray of stringThe names of the columns that comprise a dimension. They should have a database index for efficient filtering. We do not discover dimensions based on hypertable schema because they are not necessarily configured properly.
channelNameComponentsarray ofPivotedTimescaleChannelNameComponentchannelNameComponents will be combined, together with separator, to form a fully qualified channel name.
separatorstring required
Set
typeto"visualCrossing".visualCrossingVisualCrossingScrapingConfig
Fields of
VisualCrossingScrapingConfigavailableSeriesarray ofVisualCrossingAvailableSeriesdeprecatedlist of available channels Deprecated: No longer used for anything
Fields of
VisualCrossingAvailableSerieschannelstring required
unitstring
descriptionstring
typeVisualCrossingTyperequired
Values of
VisualCrossingTypeDOUBLE
STRING
Set
typeto"bigQuery".bigQueryBigQueryScrapingConfig
Fields of
BigQueryScrapingConfigtimeColumnstring requiredThe name of the column that holds the timestamp.
tagColumnsarray of stringThe name of the columns that should be interpreted as tag columns
channelNameComponentsBigQueryChannelNameComponentchannelNameComponents will be combined, together with separator, to form a fully qualified channel name. By default, we don’t add anything to the column name.
Fields of
BigQueryChannelNameComponentincludeProjectboolean requiredWhether to include the project name in the channel name.
includeDatasetboolean requiredWhether to include the dataset name in the channel name.
includeTableboolean requiredWhether to include the table name in the channel name.
separatorstringThe separator that delimits the parts of the channel name. If ommitted, the default is a “.”.
Set
typeto"api".apiApiScrapingConfig
Set
typeto"allChannels".allChannelsAllChannelsConnectionsScrapingConfig
Set
typeto"channelAllowList".channelAllowListChannelAllowListConnectionsScrapingConfig
Fields of
ChannelAllowListConnectionsScrapingConfigchannelAllowListarray of stringOnly channels in this set will be scraped. Must not be empty.
Set
typeto"channelBlockList".channelBlockListChannelBlockListConnectionsScrapingConfig
Fields of
ChannelBlockListConnectionsScrapingConfigchannelBlockListarray of stringOnly channels not in this set will be scraped. Must not be empty.
shouldScrapeboolean required
limitsLimitsConfig
Fields of
LimitsConfigmaxQueryRangeconnection.api.Duration
Fields of
connection.api.DurationunitTemporalUnitrequired
Values of
TemporalUnitHOURS
DAYS
WEEKS
countinteger required
Response
ridstring (RID)
uuidstring (UUID)
namestring
descriptionstring
displayNamestring
metadatamap of string to string deprecatedDeprecated: Deprecated in favor of properties.
handleHandledeprecatedDeprecated: Deprecated. Use DatasetFile#handle
Set
typeto"s3".s3S3Handle
Fields of
S3Handlebucketstring
keystring
ingestDatestring (date-time)Dataset creation time. The name is historical; use lastIngestedAt for when data was last added.
lastIngestedAtstring (date-time)When data was last added to this dataset, as the latest of a successful file ingest and streaming-session activity. Empty when the dataset has never received either.
ingestStatusIngestStatusdeprecatedDeprecated: Deprecated. Use lastIngestStatus.
Values of
IngestStatusIN_PROGRESS
COMPLETED
FAILED
DELETION_IN_PROGRESS
DELETED
originMetadataDatasetOriginMetadata
Fields of
DatasetOriginMetadatapathstringDeprecated in favor of FileOriginMetadata
xSeriesIsAbsolutebooleanDeprecated in favor of FileOriginMetadata
schemaDirectivePathstringDeprecated in favor of FileOriginMetadata
xSeriesColumnNamestringDeprecated in favor of FileOriginMetadata
xSeriesTimeUnitTimeUnitDeprecated in favor of FileOriginMetadata
Values of
TimeUnitDAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
timestampMetadataTimestampMetadataDeprecated in favor of FileOriginMetadata
Fields of
TimestampMetadataseriesNamestring
timestampTypecatalog.TimestampType
channelConfigChannelConfig
Fields of
ChannelConfigprefixTreeDelimiterstringIf set, will construct a prefix tree for channels of the dataset using the given delimiter.
lastIngestStatusIngestStatusV2
Set
typeto"success".successSuccessResult
Set
typeto"error".errorErrorResult
Fields of
ErrorResulterrorTypestring
messagestring
errorInstanceIdstring
Set
typeto"inProgress".inProgressInProgressResult
Fields of
InProgressResultinProgressDetailsInProgressDetails
Set
typeto"deletionInProgress".deletionInProgressDeletionInProgress
Set
typeto"deleted".deletedDeleted
Set
typeto"queued".queuedQueued
Set
typeto"parsing".parsingParsing
Set
typeto"ingesting".ingestingIngesting
Fields of
IngestingprogressnumberOptional fraction between 0 and 1 indicating progress through ingestion.
retentionPolicyRetentionPolicy
Fields of
RetentionPolicytypeRetentionPolicyType
Values of
RetentionPolicyTypeKEEP_UNTIL_EXPIRY
KEEP_FOREVER
expiryDaysinteger
sourcestring
boundsBounds
Fields of
BoundsstartTimestamp
Fields of
Timestampsecondsinteger (int64)
nanosinteger (int64)
endTimestamp
typedatasource.TimestampType
Values of
datasource.TimestampTypeRELATIVE
ABSOLUTE
timestampTypeWeakTimestampType
Values of
WeakTimestampTypeABSOLUTE
RELATIVE
PENDING
labelsarray of string
propertiesmap ofPropertyNameto string deprecatedDeprecated: use typedProperties
typedPropertiesmap ofPropertyNametoTypedPropertyValue
Set
typeto"stringValue".stringValuestring
Set
typeto"numericValue".numericValuenumber
granularityGranularity
Values of
GranularityNANOSECONDS
allowStreamingboolean
isArchivedboolean
defaultGroupByAndAggregationConfigDefaultGroupByAndAggregationConfig
Fields of
DefaultGroupByAndAggregationConfignumericAggregationNumericAggregationOperatorIf set, the default aggregation applied to merge points of a numeric series that share an exact timestamp.
Set
typeto"sum".sumSummation
Set
typeto"average".averageAverage
Set
typeto"min".minMinimum
Set
typeto"max".maxMaximum
Set
typeto"count".countCount
Set
typeto"standardDeviation".standardDeviationStandardDeviation
Set
typeto"rootMeanSquare".rootMeanSquareRootMeanSquare
Set
typeto"percentile".percentilePercentile
Set
typeto"udf".
migratedFromConnectionRidstring (RID)Connection RID whose data source was migrated into this dataset. Empty for datasets that were not created by the legacy connection-to-dataset migrations, or when the original connection row no longer exists.
externalConnectionMetadataExternalConnectionMetadata
Fields of
ExternalConnectionMetadataconnectionDetailsConnectionDetails
Set
typeto"timescale".timescaleTimescaleConnectionDetails
Set
typeto"influx".influxInflux2ConnectionDetails
Set
typeto"influx1".influx1Influx1ConnectionDetails
Set
typeto"nominal".nominalNominalConnectionDetails
Set
typeto"timestream".timestreamTimestreamConnectionDetails
Set
typeto"visualCrossing".visualCrossingVisualCrossingConnectionDetails
Set
typeto"bigQuery".bigQueryBigQueryConnectionDetails
connectionStatusConnectionStatus
Fields of
ConnectionStatusisConnectedboolean
limitsLimitsConfig
Fields of
LimitsConfigmaxQueryRangeconnection.api.Duration
derivedDefinitionDerivedDefinitionDefinition for a derived dataset. When present, this dataset is virtual and backed by the referenced provided Dataset definition instead of persisted dataset files.
Fields of
DerivedDefinitionspecapi.Dataset
Set
typeto"asset".assetAssetSingle asset. Expands to one grouping per data scope on the asset. Groupings are tagged by
assetRidanddataScopeso callers can safely compose multiple assets withcombine.
Set
typeto"run".runRunSingle run. Expands to one grouping per data source attached to the run: asset-typed sources contribute one grouping per data scope on the asset; dataset-typed sources contribute one grouping per dataset. Groupings are tagged by
runRidanddataScope; asset-typed groupings additionally carryassetRidso callers can safely compose multiple runs withcombine.
Set
typeto"saved".savedSavedDataset
Set
typeto"search".searchSearchDataset
Set
typeto"combine".combineCombinedDataset
Set
typeto"tag".tagTaggedDataset
Set
typeto"filter".filterFilteredDataset
Set
typeto"timeShift".timeShiftTimeShiftedDataset
Set
typeto"alignByAnchor".alignByAnchorAnchorAlignedDataset
Set
typeto"withSeries".withSeriesWithSeriesDataset
Set
typeto"dropSeries".dropSeriesDropSeriesDataset
Set
typeto"reference".referenceDatasetReference
Set
typeto"withDimensions".withDimensionsDimensionsDataset
commitCommit
Fields of
Commitidstring
resourceRidstring (RID)
parentCommitstring
mergeParentCommitstring
messagestring
isWorkingStatebooleanA working state commit is created via the
saveWorkingStateendpoint and is non-permanent. In the future, it may be compacted and not exist.committedBystring (RID)
committedAtstring (date-time)
channelSearchSplitTagKeysarray of stringTag keys whose distinct values channel search surfaces as separate series. Each listed key is retained on this dataset’s channel-search locators, so a channel carrying two values of that key appears once per value instead of collapsing into a single entry — the behavior a data scope has on an asset. Keys not listed here stay available for filtering and grouping but do not contribute to series cardinality. Every key must be produced by this dataset’s
derivedDefinition. Ingested tag keys cannot be split on.
Request
curl -X POST "https://api.gov.nominal.io/api/catalog/v1/datasets/external-config" \
-H "Authorization: Bearer <api key>" \
-H "Content-Type: application/json" \
--data '{
"datasetRid": "<dataset rid>",
"connectionDetails": {
"type": "timescale",
"timescale": {
"database": "<database>",
"host": "<host>",
"port": 1,
"username": "<username>",
"passwordSecretRid": "<secret rid>"
}
},
"shouldScrape": true
}'
import requests
response = requests.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/external-config",
headers={"Authorization": "Bearer <api key>"},
json={
"datasetRid": "<dataset rid>",
"connectionDetails": {
"type": "timescale",
"timescale": {
"database": "<database>",
"host": "<host>",
"port": 1,
"username": "<username>",
"passwordSecretRid": "<secret rid>",
},
},
"shouldScrape": True,
},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/external-config",
headers={"Authorization": "Bearer <api key>"},
json={
"datasetRid": "<dataset rid>",
"connectionDetails": {
"type": "timescale",
"timescale": {
"database": "<database>",
"host": "<host>",
"port": 1,
"username": "<username>",
"passwordSecretRid": "<secret rid>",
},
},
"shouldScrape": True,
},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/external-config",
headers={"Authorization": "Bearer <api key>"},
json={
"datasetRid": "<dataset rid>",
"connectionDetails": {
"type": "timescale",
"timescale": {
"database": "<database>",
"host": "<host>",
"port": 1,
"username": "<username>",
"passwordSecretRid": "<secret rid>",
},
},
"shouldScrape": True,
},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"rid": "<rid>",
"uuid": "<uuid>",
"name": "<name>",
"description": "<description>",
"displayName": "<display name>",
"metadata": {
"<key>": "<value>"
},
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"ingestDate": "2026-01-01T00:00:00Z",
"lastIngestedAt": "2026-01-01T00:00:00Z",
"ingestStatus": "IN_PROGRESS",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": true,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS",
"offset": "2026-01-01T00:00:00Z"
}
}
},
"channelConfig": {
"prefixTreeDelimiter": "<prefix tree delimiter>"
}
},
"lastIngestStatus": {
"type": "success",
"success": {}
},
"retentionPolicy": {
"type": "KEEP_UNTIL_EXPIRY",
"expiryDays": 1
},
"source": "<source>",
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"timestampType": "ABSOLUTE",
"labels": [
"<label>"
],
"properties": {
"<key>": "<property value>"
},
"typedProperties": {
"<key>": {
"type": "stringValue",
"stringValue": "<property value>"
}
},
"granularity": "NANOSECONDS",
"allowStreaming": true,
"isArchived": true,
"defaultGroupByAndAggregationConfig": {
"numericAggregation": {
"type": "sum",
"sum": {}
}
},
"migratedFromConnectionRid": "<connection rid>",
"externalConnectionMetadata": {
"connectionDetails": {
"type": "timescale",
"timescale": {
"database": "<database>",
"host": "<host>",
"port": 1,
"username": "<username>",
"passwordSecretRid": "<secret rid>"
}
},
"connectionStatus": {
"isConnected": true
},
"limits": {
"maxQueryRange": {
"unit": "HOURS",
"count": 1
}
}
},
"derivedDefinition": {
"spec": {
"type": "asset",
"asset": {
"rid": {
"type": "literal",
"literal": "<literal>"
}
}
},
"commit": {
"id": "<commit id>",
"resourceRid": "<resource rid>",
"parentCommit": "<commit id>",
"mergeParentCommit": "<commit id>",
"message": "<message>",
"isWorkingState": true,
"committedBy": "<user rid>",
"committedAt": "2026-01-01T00:00:00Z"
}
},
"channelSearchSplitTagKeys": [
"<tag name>"
]
}
Get dataset file#
GET /catalog/v1/dataset/{datasetRid}/file/{fileId}
Path parameters
datasetRidstring (RID) required
fileIdstring (UUID) required
Response
idstring (UUID)
datasetRidstring (RID)
namestring
handleHandle
Set
typeto"s3".s3S3Handle
Fields of
S3Handlebucketstring
keystring
fileSizeBytesinteger (int64)Size in bytes of the stored dataset file; for transforming ingests this is the produced output, not the original upload.
boundsBounds
Fields of
BoundsstartTimestamp
Fields of
Timestampsecondsinteger (int64)
nanosinteger (int64)
endTimestamp
typedatasource.TimestampType
Values of
datasource.TimestampTypeRELATIVE
ABSOLUTE
uploadedAtstring (date-time)Timestamp that the file was received and stored, but not processed or made available to consumers.
ingestedAtstring (date-time)Timestamp that the file is ingested at and made available for processing. If the file has failed to be ingested for any reason or is still being processed, then this value will be empty.
ingestStatusIngestStatusV2Ingest status. The in-progress sub-phases (queued, parsing, ingesting) are surfaced as inProgress with the phase carried in InProgressResult.inProgressDetails, so consumers see a stable top-level lifecycle.
Set
typeto"success".successSuccessResult
Set
typeto"error".errorErrorResult
Fields of
ErrorResulterrorTypestring
messagestring
errorInstanceIdstring
Set
typeto"inProgress".inProgressInProgressResult
Fields of
InProgressResultinProgressDetailsInProgressDetails
Set
typeto"deletionInProgress".deletionInProgressDeletionInProgress
Set
typeto"deleted".deletedDeleted
Set
typeto"queued".queuedQueued
Set
typeto"parsing".parsingParsing
Set
typeto"ingesting".ingestingIngesting
timestampMetadataTimestampMetadata
Fields of
TimestampMetadataseriesNamestring
timestampTypecatalog.TimestampType
Set
typeto"relative".relativeRelativeTimestamp
Fields of
RelativeTimestamptimeUnitTimeUnit
Values of
TimeUnitDAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
offsetstring (date-time)
Set
typeto"absolute".absoluteAbsoluteTimestamp
Set
typeto"iso8601".iso8601Iso8601Timestamp
Set
typeto"epochOfTimeUnit".epochOfTimeUnitEpochTimestamp
Fields of
EpochTimestamptimeUnitTimeUnit
Set
typeto"customFormat".customFormatCustomTimestamp
Fields of
CustomTimestampformatstringThe format string should be in the format of the
DateTimeFormatterclass in Java.defaultYearintegerYear is accepted as an optional field for cases like IRIG time format, and will be assumed as current year if not provided.
defaultDayOfYearintegerDefault day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.
ingestTagMetadataIngestTagMetadata
Fields of
IngestTagMetadatatagColumnsmap ofTagNameto stringA map of tag names to column names to derive the tag values from.
additionalFileTagsmap ofTagNameto string
originFilePathsarray of string
ingestJobRidstring (RID)
ingestTransformRidstring (RID)The ingest transform within ingestJobRid that produced this file. An ingest job may fan out to many transforms, so this pins the file to the specific unit of work (and extractor) that created it — resolvable via IngestJobService#getIngestTransform. Extractor logs are tagged with this transform’s UUID (the RID locator), so this also links a file to the logs of the transform that produced it. Absent for directly uploaded files and for files ingested before this link was recorded.
deletedAtstring (date-time)Timestamp that the file is deleted at, only present if the file has been deleted.
metadataDatasetFileMetadataFile-type-specific metadata. For video files, contains timestamp manifest and segment metadata.
Set
typeto"video".videoVideoFileMetadata
Fields of
VideoFileMetadatatimestampManifestVideoFileTimestampManifestSpecifies how to determine absolute timestamps for each frame in the video. Can be an embedded MCAP manifest, an external sidecar file, or calculated from a starting offset applied to presentation timestamps.
Set
typeto"mcap".mcapMcapTimestampManifest
Fields of
McapTimestampManifestmcapChannelLocatorMcapChannelLocator
Set
typeto"topic".topicstring
Set
typeto"id".idinteger
Set
typeto"s3path".s3pathstring
Set
typeto"s3paths".s3pathsarray of stringMultiple sidecar manifest files whose per-frame timestamps are concatenated in list order. Used when a video’s frame timestamps are recorded across several files.
Set
typeto"noManifest".noManifestNoTimestampManifest
Fields of
NoTimestampManifeststartingTimestampTimestampSpecifies the original starting timestamp of the video.
scaleParameterScaleParameterA field that specifies that the frame rate of the video does not match the frame rate of the camera | i.e. a slowed down or sped up video. Can specify either the camera frame rate or the absolute end time.
Set
typeto"trueFrameRate".trueFrameRatenumber
Set
typeto"endingTimestamp".endingTimestampTimestamp
Set
typeto"scaleFactor".scaleFactornumberthe scale factor can be used to calculate whether media duration differs from a video’s | real duration, and if so, the true frame rate of the camera. The video time will thus be scaled | by the ratio of the real duration to media duration, or media frame rate to true frame rate.
segmentMetadataVideoSegmentsMetadataCached aggregate metadata about the segments comprising this video file after segmentation has completed. Includes frame counts, duration, and frame rate. Empty until segmentation is complete. Note: Min/max timestamps are stored in DatasetFile.bounds field. Raw file size is stored in DatasetFile.fileSizeBytes.
Fields of
VideoSegmentsMetadatanumFramesintegerTotal number of frames across all segments.
numSegmentsintegerNumber of segments the video was split into.
scaleFactornumberScale factor applied to timestamps during segmentation.
mediaDurationSecondsnumberTotal duration of the video in seconds.
mediaFrameRatenumberAverage frame rate (FPS) calculated as total frames / duration.
Request
curl "https://api.gov.nominal.io/api/catalog/v1/dataset/<dataset rid>/file/<file id>" \
-H "Authorization: Bearer <api key>"
import requests
response = requests.get(
"https://api.gov.nominal.io/api/catalog/v1/dataset/<dataset rid>/file/<file id>",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.get(
"https://api.gov.nominal.io/api/catalog/v1/dataset/<dataset rid>/file/<file id>",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.get(
"https://api.gov.nominal.io/api/catalog/v1/dataset/<dataset rid>/file/<file id>",
headers={"Authorization": "Bearer <api key>"},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"id": "<dataset file id>",
"datasetRid": "<dataset rid>",
"name": "<name>",
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"fileSizeBytes": 1,
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"uploadedAt": "2026-01-01T00:00:00Z",
"ingestedAt": "2026-01-01T00:00:00Z",
"ingestStatus": {
"type": "success",
"success": {}
},
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS",
"offset": "2026-01-01T00:00:00Z"
}
}
},
"ingestTagMetadata": {
"tagColumns": {
"<key>": "<column name>"
},
"additionalFileTags": {
"<key>": "<tag value>"
}
},
"originFilePaths": [
"<origin file paths>"
],
"ingestJobRid": "<ingest job rid>",
"ingestTransformRid": "<ingest transform rid>",
"deletedAt": "2026-01-01T00:00:00Z",
"metadata": {
"type": "video",
"video": {
"timestampManifest": {
"type": "mcap",
"mcap": {
"mcapChannelLocator": {
"type": "topic",
"topic": "<mcap channel topic>"
}
}
},
"segmentMetadata": {
"numFrames": 1,
"numSegments": 1,
"scaleFactor": 1,
"mediaDurationSeconds": 1,
"mediaFrameRate": 1
}
}
}
}
Batch get dataset files#
POST /catalog/v1/dataset-files/batchGet
Returns dataset files for the given file IDs within a single dataset. Only returns files that exist and belong to the specified dataset. Useful for checking ingestion status of many files at once.
Body
datasetRidstring (RID) required
fileIdsarray of string (UUID)
Response
The body is a map of DatasetFileId to DatasetFile.
Fields of DatasetFile
idstring (UUID)
datasetRidstring (RID)
namestring
handleHandle
Set
typeto"s3".s3S3Handle
Fields of
S3Handlebucketstring
keystring
fileSizeBytesinteger (int64)Size in bytes of the stored dataset file; for transforming ingests this is the produced output, not the original upload.
boundsBounds
Fields of
BoundsstartTimestamp
Fields of
Timestampsecondsinteger (int64)
nanosinteger (int64)
endTimestamp
typedatasource.TimestampType
Values of
datasource.TimestampTypeRELATIVE
ABSOLUTE
uploadedAtstring (date-time)Timestamp that the file was received and stored, but not processed or made available to consumers.
ingestedAtstring (date-time)Timestamp that the file is ingested at and made available for processing. If the file has failed to be ingested for any reason or is still being processed, then this value will be empty.
ingestStatusIngestStatusV2Ingest status. The in-progress sub-phases (queued, parsing, ingesting) are surfaced as inProgress with the phase carried in InProgressResult.inProgressDetails, so consumers see a stable top-level lifecycle.
Set
typeto"success".successSuccessResult
Set
typeto"error".errorErrorResult
Fields of
ErrorResulterrorTypestring
messagestring
errorInstanceIdstring
Set
typeto"inProgress".inProgressInProgressResult
Fields of
InProgressResultinProgressDetailsInProgressDetails
Set
typeto"deletionInProgress".deletionInProgressDeletionInProgress
Set
typeto"deleted".deletedDeleted
Set
typeto"queued".queuedQueued
Set
typeto"parsing".parsingParsing
Set
typeto"ingesting".ingestingIngesting
timestampMetadataTimestampMetadata
Fields of
TimestampMetadataseriesNamestring
timestampTypecatalog.TimestampType
Set
typeto"relative".relativeRelativeTimestamp
Fields of
RelativeTimestamptimeUnitTimeUnit
Values of
TimeUnitDAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
offsetstring (date-time)
Set
typeto"absolute".absoluteAbsoluteTimestamp
Set
typeto"iso8601".iso8601Iso8601Timestamp
Set
typeto"epochOfTimeUnit".epochOfTimeUnitEpochTimestamp
Fields of
EpochTimestamptimeUnitTimeUnit
Set
typeto"customFormat".customFormatCustomTimestamp
Fields of
CustomTimestampformatstringThe format string should be in the format of the
DateTimeFormatterclass in Java.defaultYearintegerYear is accepted as an optional field for cases like IRIG time format, and will be assumed as current year if not provided.
defaultDayOfYearintegerDefault day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.
ingestTagMetadataIngestTagMetadata
Fields of
IngestTagMetadatatagColumnsmap ofTagNameto stringA map of tag names to column names to derive the tag values from.
additionalFileTagsmap ofTagNameto string
originFilePathsarray of string
ingestJobRidstring (RID)
ingestTransformRidstring (RID)The ingest transform within ingestJobRid that produced this file. An ingest job may fan out to many transforms, so this pins the file to the specific unit of work (and extractor) that created it — resolvable via IngestJobService#getIngestTransform. Extractor logs are tagged with this transform’s UUID (the RID locator), so this also links a file to the logs of the transform that produced it. Absent for directly uploaded files and for files ingested before this link was recorded.
deletedAtstring (date-time)Timestamp that the file is deleted at, only present if the file has been deleted.
metadataDatasetFileMetadataFile-type-specific metadata. For video files, contains timestamp manifest and segment metadata.
Set
typeto"video".videoVideoFileMetadata
Fields of
VideoFileMetadatatimestampManifestVideoFileTimestampManifestSpecifies how to determine absolute timestamps for each frame in the video. Can be an embedded MCAP manifest, an external sidecar file, or calculated from a starting offset applied to presentation timestamps.
Set
typeto"mcap".mcapMcapTimestampManifest
Fields of
McapTimestampManifestmcapChannelLocatorMcapChannelLocator
Set
typeto"topic".topicstring
Set
typeto"id".idinteger
Set
typeto"s3path".s3pathstring
Set
typeto"s3paths".s3pathsarray of stringMultiple sidecar manifest files whose per-frame timestamps are concatenated in list order. Used when a video’s frame timestamps are recorded across several files.
Set
typeto"noManifest".noManifestNoTimestampManifest
Fields of
NoTimestampManifeststartingTimestampTimestampSpecifies the original starting timestamp of the video.
scaleParameterScaleParameterA field that specifies that the frame rate of the video does not match the frame rate of the camera | i.e. a slowed down or sped up video. Can specify either the camera frame rate or the absolute end time.
Set
typeto"trueFrameRate".trueFrameRatenumber
Set
typeto"endingTimestamp".endingTimestampTimestamp
Set
typeto"scaleFactor".scaleFactornumberthe scale factor can be used to calculate whether media duration differs from a video’s | real duration, and if so, the true frame rate of the camera. The video time will thus be scaled | by the ratio of the real duration to media duration, or media frame rate to true frame rate.
segmentMetadataVideoSegmentsMetadataCached aggregate metadata about the segments comprising this video file after segmentation has completed. Includes frame counts, duration, and frame rate. Empty until segmentation is complete. Note: Min/max timestamps are stored in DatasetFile.bounds field. Raw file size is stored in DatasetFile.fileSizeBytes.
Fields of
VideoSegmentsMetadatanumFramesintegerTotal number of frames across all segments.
numSegmentsintegerNumber of segments the video was split into.
scaleFactornumberScale factor applied to timestamps during segmentation.
mediaDurationSecondsnumberTotal duration of the video in seconds.
mediaFrameRatenumberAverage frame rate (FPS) calculated as total frames / duration.
Request
curl -X POST "https://api.gov.nominal.io/api/catalog/v1/dataset-files/batchGet" \
-H "Authorization: Bearer <api key>" \
-H "Content-Type: application/json" \
--data '{
"datasetRid": "<dataset rid>"
}'
import requests
response = requests.post(
"https://api.gov.nominal.io/api/catalog/v1/dataset-files/batchGet",
headers={"Authorization": "Bearer <api key>"},
json={"datasetRid": "<dataset rid>"},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.post(
"https://api.gov.nominal.io/api/catalog/v1/dataset-files/batchGet",
headers={"Authorization": "Bearer <api key>"},
json={"datasetRid": "<dataset rid>"},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.post(
"https://api.gov.nominal.io/api/catalog/v1/dataset-files/batchGet",
headers={"Authorization": "Bearer <api key>"},
json={"datasetRid": "<dataset rid>"},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"<key>": {
"id": "<dataset file id>",
"datasetRid": "<dataset rid>",
"name": "<name>",
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"fileSizeBytes": 1,
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"uploadedAt": "2026-01-01T00:00:00Z",
"ingestedAt": "2026-01-01T00:00:00Z",
"ingestStatus": {
"type": "success",
"success": {}
},
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS",
"offset": "2026-01-01T00:00:00Z"
}
}
},
"ingestTagMetadata": {
"tagColumns": {
"<key>": "<column name>"
},
"additionalFileTags": {
"<key>": "<tag value>"
}
},
"originFilePaths": [
"<origin file paths>"
],
"ingestJobRid": "<ingest job rid>",
"ingestTransformRid": "<ingest transform rid>",
"deletedAt": "2026-01-01T00:00:00Z",
"metadata": {
"type": "video",
"video": {
"timestampManifest": {
"type": "mcap",
"mcap": {
"mcapChannelLocator": {
"type": "topic",
"topic": "<mcap channel topic>"
}
}
},
"segmentMetadata": {
"numFrames": 1,
"numSegments": 1,
"scaleFactor": 1,
"mediaDurationSeconds": 1,
"mediaFrameRate": 1
}
}
}
}
}
Get dataset files for job#
GET /catalog/v1/ingest-job/{ingestJobRid}/files
Path parameters
ingestJobRidstring (RID) required
Query parameters
nextPageTokenstring
Response
filesarray ofDatasetFile
Fields of
DatasetFileidstring (UUID)
datasetRidstring (RID)
namestring
handleHandle
Set
typeto"s3".s3S3Handle
Fields of
S3Handlebucketstring
keystring
fileSizeBytesinteger (int64)Size in bytes of the stored dataset file; for transforming ingests this is the produced output, not the original upload.
boundsBounds
Fields of
BoundsstartTimestamp
Fields of
Timestampsecondsinteger (int64)
nanosinteger (int64)
endTimestamp
typedatasource.TimestampType
Values of
datasource.TimestampTypeRELATIVE
ABSOLUTE
uploadedAtstring (date-time)Timestamp that the file was received and stored, but not processed or made available to consumers.
ingestedAtstring (date-time)Timestamp that the file is ingested at and made available for processing. If the file has failed to be ingested for any reason or is still being processed, then this value will be empty.
ingestStatusIngestStatusV2Ingest status. The in-progress sub-phases (queued, parsing, ingesting) are surfaced as inProgress with the phase carried in InProgressResult.inProgressDetails, so consumers see a stable top-level lifecycle.
Set
typeto"success".successSuccessResult
Set
typeto"error".errorErrorResult
Fields of
ErrorResulterrorTypestring
messagestring
errorInstanceIdstring
Set
typeto"inProgress".inProgressInProgressResult
Fields of
InProgressResultinProgressDetailsInProgressDetails
Set
typeto"deletionInProgress".deletionInProgressDeletionInProgress
Set
typeto"deleted".deletedDeleted
Set
typeto"queued".queuedQueued
Set
typeto"parsing".parsingParsing
Set
typeto"ingesting".ingestingIngesting
timestampMetadataTimestampMetadata
Fields of
TimestampMetadataseriesNamestring
timestampTypecatalog.TimestampType
Set
typeto"relative".relativeRelativeTimestamp
Fields of
RelativeTimestamptimeUnitTimeUnit
Values of
TimeUnitDAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
offsetstring (date-time)
Set
typeto"absolute".absoluteAbsoluteTimestamp
Set
typeto"iso8601".iso8601Iso8601Timestamp
Set
typeto"epochOfTimeUnit".epochOfTimeUnitEpochTimestamp
Fields of
EpochTimestamptimeUnitTimeUnit
Set
typeto"customFormat".customFormatCustomTimestamp
Fields of
CustomTimestampformatstringThe format string should be in the format of the
DateTimeFormatterclass in Java.defaultYearintegerYear is accepted as an optional field for cases like IRIG time format, and will be assumed as current year if not provided.
defaultDayOfYearintegerDefault day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.
ingestTagMetadataIngestTagMetadata
Fields of
IngestTagMetadatatagColumnsmap ofTagNameto stringA map of tag names to column names to derive the tag values from.
additionalFileTagsmap ofTagNameto string
originFilePathsarray of string
ingestJobRidstring (RID)
ingestTransformRidstring (RID)The ingest transform within ingestJobRid that produced this file. An ingest job may fan out to many transforms, so this pins the file to the specific unit of work (and extractor) that created it — resolvable via IngestJobService#getIngestTransform. Extractor logs are tagged with this transform’s UUID (the RID locator), so this also links a file to the logs of the transform that produced it. Absent for directly uploaded files and for files ingested before this link was recorded.
deletedAtstring (date-time)Timestamp that the file is deleted at, only present if the file has been deleted.
metadataDatasetFileMetadataFile-type-specific metadata. For video files, contains timestamp manifest and segment metadata.
Set
typeto"video".videoVideoFileMetadata
Fields of
VideoFileMetadatatimestampManifestVideoFileTimestampManifestSpecifies how to determine absolute timestamps for each frame in the video. Can be an embedded MCAP manifest, an external sidecar file, or calculated from a starting offset applied to presentation timestamps.
Set
typeto"mcap".mcapMcapTimestampManifest
Fields of
McapTimestampManifestmcapChannelLocatorMcapChannelLocator
Set
typeto"topic".topicstring
Set
typeto"id".idinteger
Set
typeto"s3path".s3pathstring
Set
typeto"s3paths".s3pathsarray of stringMultiple sidecar manifest files whose per-frame timestamps are concatenated in list order. Used when a video’s frame timestamps are recorded across several files.
Set
typeto"noManifest".noManifestNoTimestampManifest
Fields of
NoTimestampManifeststartingTimestampTimestampSpecifies the original starting timestamp of the video.
scaleParameterScaleParameterA field that specifies that the frame rate of the video does not match the frame rate of the camera | i.e. a slowed down or sped up video. Can specify either the camera frame rate or the absolute end time.
Set
typeto"trueFrameRate".trueFrameRatenumber
Set
typeto"endingTimestamp".endingTimestampTimestamp
Set
typeto"scaleFactor".scaleFactornumberthe scale factor can be used to calculate whether media duration differs from a video’s | real duration, and if so, the true frame rate of the camera. The video time will thus be scaled | by the ratio of the real duration to media duration, or media frame rate to true frame rate.
segmentMetadataVideoSegmentsMetadataCached aggregate metadata about the segments comprising this video file after segmentation has completed. Includes frame counts, duration, and frame rate. Empty until segmentation is complete. Note: Min/max timestamps are stored in DatasetFile.bounds field. Raw file size is stored in DatasetFile.fileSizeBytes.
Fields of
VideoSegmentsMetadatanumFramesintegerTotal number of frames across all segments.
numSegmentsintegerNumber of segments the video was split into.
scaleFactornumberScale factor applied to timestamps during segmentation.
mediaDurationSecondsnumberTotal duration of the video in seconds.
mediaFrameRatenumberAverage frame rate (FPS) calculated as total frames / duration.
nextPagestring
Request
curl "https://api.gov.nominal.io/api/catalog/v1/ingest-job/<ingest job rid>/files" \
-H "Authorization: Bearer <api key>"
import requests
response = requests.get(
"https://api.gov.nominal.io/api/catalog/v1/ingest-job/<ingest job rid>/files",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.get(
"https://api.gov.nominal.io/api/catalog/v1/ingest-job/<ingest job rid>/files",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.get(
"https://api.gov.nominal.io/api/catalog/v1/ingest-job/<ingest job rid>/files",
headers={"Authorization": "Bearer <api key>"},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"files": [
{
"id": "<dataset file id>",
"datasetRid": "<dataset rid>",
"name": "<name>",
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"fileSizeBytes": 1,
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"uploadedAt": "2026-01-01T00:00:00Z",
"ingestedAt": "2026-01-01T00:00:00Z",
"ingestStatus": {
"type": "success",
"success": {}
},
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS",
"offset": "2026-01-01T00:00:00Z"
}
}
},
"ingestTagMetadata": {
"tagColumns": {
"<key>": "<column name>"
},
"additionalFileTags": {
"<key>": "<tag value>"
}
},
"originFilePaths": [
"<origin file paths>"
],
"ingestJobRid": "<ingest job rid>",
"ingestTransformRid": "<ingest transform rid>",
"deletedAt": "2026-01-01T00:00:00Z",
"metadata": {
"type": "video",
"video": {
"timestampManifest": {
"type": "mcap",
"mcap": {
"mcapChannelLocator": {
"type": "topic",
"topic": "<mcap channel topic>"
}
}
},
"segmentMetadata": {
"numFrames": 1,
"numSegments": 1,
"scaleFactor": 1,
"mediaDurationSeconds": 1,
"mediaFrameRate": 1
}
}
}
}
],
"nextPage": "<token>"
}
Search datasets#
POST /catalog/v1/search-datasets-v2
Body
querySearchDatasetsQueryrequired
Set
typeto"searchText".searchTextstring
Set
typeto"exactMatch".exactMatchstringPerforms case insensitive exact match search on the name.
Set
typeto"label".labelstring
Set
typeto"properties".propertiesnominal.api.Property
Fields of
nominal.api.Propertynamestring required
valuestring required
Set
typeto"numericProperty".numericPropertyNumericPropertyPredicateMatches datasets using a numeric property comparison.
Fields of
NumericPropertyPredicatenamestring required
valuenumber required
operatorPropertyComparisonOperatorrequired
Values of
PropertyComparisonOperatorEQ
NEQ
GT
GTE
LT
LTE
Set
typeto"numericPropertyRange".numericPropertyRangeNumericPropertyRangePredicateMatches datasets using an inclusive numeric property range.
Fields of
NumericPropertyRangePredicatenamestring required
minnumber
maxnumber
operatorNumericPropertyRangeOperatorrequired
Values of
NumericPropertyRangeOperatorBETWEEN
NOT_BETWEEN
Set
typeto"ingestStatus".ingestStatusIngestStatus
Values of
IngestStatusIN_PROGRESS
COMPLETED
FAILED
DELETION_IN_PROGRESS
DELETED
Set
typeto"ingestedBeforeInclusive".ingestedBeforeInclusivestring (date-time)Inclusive upper bound on dataset creation time (ingestDate), not lastIngestedAt.
Set
typeto"ingestedAfterInclusive".ingestedAfterInclusivestring (date-time)Inclusive lower bound on dataset creation time (ingestDate), not lastIngestedAt.
Set
typeto"archiveStatus".archiveStatusboolean
Set
typeto"derived".derivedbooleanFilters datasets by whether they have a derived definition.
Set
typeto"and".andarray ofSearchDatasetsQuery
Set
typeto"or".orarray ofSearchDatasetsQuery
Set
typeto"workspace".workspacestring (RID)
pageSizeintegerDefaults to 100. Will throw if larger than 1000.
tokenstring
sortOptionsSortOptionsrequired
Fields of
SortOptionsisDescendingboolean required
fieldSortFieldrequired
Values of
SortFieldINGEST_DATEDataset creation time (created_at). The name is historical; use LAST_INGESTED_AT for when data was last added.
START_TIMEStart of the dataset’s data bounds.
END_TIMEEnd of the dataset’s data bounds.
LAST_INGESTED_ATWhen data was last added, as the latest of a successful file ingest and streaming-session activity. Datasets that have never received either sort last.
Response
resultsarray ofEnrichedDataset
Fields of
EnrichedDatasetridstring (RID)
uuidstring (UUID)
namestring
descriptionstring
displayNamestring
metadatamap of string to string deprecatedDeprecated: Deprecated in favor of properties.
handleHandledeprecatedDeprecated: Deprecated. Use DatasetFile#handle
Set
typeto"s3".s3S3Handle
ingestDatestring (date-time)Dataset creation time. The name is historical; use lastIngestedAt for when data was last added.
lastIngestedAtstring (date-time)When data was last added to this dataset, as the latest of a successful file ingest and streaming-session activity. Empty when the dataset has never received either.
ingestStatusIngestStatusdeprecatedDeprecated: Deprecated. Use lastIngestStatus.
Values of
IngestStatusIN_PROGRESS
COMPLETED
FAILED
DELETION_IN_PROGRESS
DELETED
originMetadataDatasetOriginMetadata
Fields of
DatasetOriginMetadatapathstringDeprecated in favor of FileOriginMetadata
xSeriesIsAbsolutebooleanDeprecated in favor of FileOriginMetadata
schemaDirectivePathstringDeprecated in favor of FileOriginMetadata
xSeriesColumnNamestringDeprecated in favor of FileOriginMetadata
xSeriesTimeUnitTimeUnitDeprecated in favor of FileOriginMetadata
timestampMetadataTimestampMetadataDeprecated in favor of FileOriginMetadata
channelConfigChannelConfig
lastIngestStatusIngestStatusV2
Set
typeto"success".successSuccessResult
Set
typeto"error".errorErrorResult
Set
typeto"inProgress".inProgressInProgressResult
Set
typeto"deletionInProgress".deletionInProgressDeletionInProgress
Set
typeto"deleted".deletedDeleted
Set
typeto"queued".queuedQueued
Set
typeto"parsing".parsingParsing
Set
typeto"ingesting".ingestingIngesting
retentionPolicyRetentionPolicy
Fields of
RetentionPolicytypeRetentionPolicyType
expiryDaysinteger
sourcestring
boundsBounds
Fields of
BoundsstartTimestamp
endTimestamp
typedatasource.TimestampType
timestampTypeWeakTimestampType
Values of
WeakTimestampTypeABSOLUTE
RELATIVE
PENDING
labelsarray of string
propertiesmap ofPropertyNameto string deprecatedDeprecated: use typedProperties
typedPropertiesmap ofPropertyNametoTypedPropertyValue
Set
typeto"stringValue".stringValuestring
Set
typeto"numericValue".numericValuenumber
granularityGranularity
Values of
GranularityNANOSECONDS
allowStreamingboolean
isArchivedboolean
defaultGroupByAndAggregationConfigDefaultGroupByAndAggregationConfig
Fields of
DefaultGroupByAndAggregationConfignumericAggregationNumericAggregationOperatorIf set, the default aggregation applied to merge points of a numeric series that share an exact timestamp.
migratedFromConnectionRidstring (RID)Connection RID whose data source was migrated into this dataset. Empty for datasets that were not created by the legacy connection-to-dataset migrations, or when the original connection row no longer exists.
externalConnectionMetadataExternalConnectionMetadata
Fields of
ExternalConnectionMetadataconnectionDetailsConnectionDetails
connectionStatusConnectionStatus
limitsLimitsConfig
derivedDefinitionDerivedDefinitionDefinition for a derived dataset. When present, this dataset is virtual and backed by the referenced provided Dataset definition instead of persisted dataset files.
Fields of
DerivedDefinitionspecapi.Dataset
commitCommit
channelSearchSplitTagKeysarray of stringTag keys whose distinct values channel search surfaces as separate series. Each listed key is retained on this dataset’s channel-search locators, so a channel carrying two values of that key appears once per value instead of collapsing into a single entry — the behavior a data scope has on an asset. Keys not listed here stay available for filtering and grouping but do not contribute to series cardinality. Every key must be produced by this dataset’s
derivedDefinition. Ingested tag keys cannot be split on.
nextPageTokenstring
Request
curl -X POST "https://api.gov.nominal.io/api/catalog/v1/search-datasets-v2" \
-H "Authorization: Bearer <api key>" \
-H "Content-Type: application/json" \
--data '{
"query": {
"type": "searchText",
"searchText": "<search text>"
},
"sortOptions": {
"isDescending": true,
"field": "INGEST_DATE"
}
}'
import requests
response = requests.post(
"https://api.gov.nominal.io/api/catalog/v1/search-datasets-v2",
headers={"Authorization": "Bearer <api key>"},
json={
"query": {"type": "searchText", "searchText": "<search text>"},
"sortOptions": {"isDescending": True, "field": "INGEST_DATE"},
},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.post(
"https://api.gov.nominal.io/api/catalog/v1/search-datasets-v2",
headers={"Authorization": "Bearer <api key>"},
json={
"query": {"type": "searchText", "searchText": "<search text>"},
"sortOptions": {"isDescending": True, "field": "INGEST_DATE"},
},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.post(
"https://api.gov.nominal.io/api/catalog/v1/search-datasets-v2",
headers={"Authorization": "Bearer <api key>"},
json={
"query": {"type": "searchText", "searchText": "<search text>"},
"sortOptions": {"isDescending": True, "field": "INGEST_DATE"},
},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"results": [
{
"rid": "<rid>",
"uuid": "<uuid>",
"name": "<name>",
"description": "<description>",
"displayName": "<display name>",
"metadata": {
"<key>": "<value>"
},
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"ingestDate": "2026-01-01T00:00:00Z",
"lastIngestedAt": "2026-01-01T00:00:00Z",
"ingestStatus": "IN_PROGRESS",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": true,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS",
"offset": "2026-01-01T00:00:00Z"
}
}
},
"channelConfig": {
"prefixTreeDelimiter": "<prefix tree delimiter>"
}
},
"lastIngestStatus": {
"type": "success",
"success": {}
},
"retentionPolicy": {
"type": "KEEP_UNTIL_EXPIRY",
"expiryDays": 1
},
"source": "<source>",
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"timestampType": "ABSOLUTE",
"labels": [
"<label>"
],
"properties": {
"<key>": "<property value>"
},
"typedProperties": {
"<key>": {
"type": "stringValue",
"stringValue": "<property value>"
}
},
"granularity": "NANOSECONDS",
"allowStreaming": true,
"isArchived": true,
"defaultGroupByAndAggregationConfig": {
"numericAggregation": {
"type": "sum",
"sum": {}
}
},
"migratedFromConnectionRid": "<connection rid>",
"externalConnectionMetadata": {
"connectionDetails": {
"type": "timescale",
"timescale": {
"database": "<database>",
"host": "<host>",
"port": 1,
"username": "<username>",
"passwordSecretRid": "<secret rid>"
}
},
"connectionStatus": {
"isConnected": true
},
"limits": {
"maxQueryRange": {
"unit": "HOURS",
"count": 1
}
}
},
"derivedDefinition": {
"spec": {
"type": "asset",
"asset": {
"rid": {
"type": "literal",
"literal": "<literal>"
}
}
},
"commit": {
"id": "<commit id>",
"resourceRid": "<resource rid>",
"parentCommit": "<commit id>",
"mergeParentCommit": "<commit id>",
"message": "<message>",
"isWorkingState": true,
"committedBy": "<user rid>",
"committedAt": "2026-01-01T00:00:00Z"
}
},
"channelSearchSplitTagKeys": [
"<tag name>"
]
}
],
"nextPageToken": "<token>"
}
Update dataset ingest status v2#
PUT /catalog/v1/datasets/ingest-status-v2
Body
statusIngestStatusV2required
Set
typeto"success".successSuccessResult
Set
typeto"error".errorErrorResult
Fields of
ErrorResulterrorTypestring required
messagestring required
errorInstanceIdstring
Set
typeto"inProgress".inProgressInProgressResult
Fields of
InProgressResultinProgressDetailsInProgressDetails
Set
typeto"deletionInProgress".deletionInProgressDeletionInProgress
Set
typeto"deleted".deletedDeleted
Set
typeto"queued".queuedQueued
Set
typeto"parsing".parsingParsing
Set
typeto"ingesting".ingestingIngesting
datasetUuidstring (UUID) required
Response
A IngestStatusV2 union.
Set type to "success".
successSuccessResult
Set type to "error".
errorErrorResult
Fields of
ErrorResulterrorTypestring
messagestring
errorInstanceIdstring
Set type to "inProgress".
inProgressInProgressResult
Fields of
InProgressResultinProgressDetailsInProgressDetails
Set type to "deletionInProgress".
deletionInProgressDeletionInProgress
Set type to "deleted".
deletedDeleted
Set type to "queued".
queuedQueued
Set type to "parsing".
parsingParsing
Set type to "ingesting".
ingestingIngesting
Request
curl -X PUT "https://api.gov.nominal.io/api/catalog/v1/datasets/ingest-status-v2" \
-H "Authorization: Bearer <api key>" \
-H "Content-Type: application/json" \
--data '{
"status": {
"type": "success",
"success": {}
},
"datasetUuid": "<dataset uuid>"
}'
import requests
response = requests.put(
"https://api.gov.nominal.io/api/catalog/v1/datasets/ingest-status-v2",
headers={"Authorization": "Bearer <api key>"},
json={"status": {"type": "success", "success": {}}, "datasetUuid": "<dataset uuid>"},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.put(
"https://api.gov.nominal.io/api/catalog/v1/datasets/ingest-status-v2",
headers={"Authorization": "Bearer <api key>"},
json={"status": {"type": "success", "success": {}}, "datasetUuid": "<dataset uuid>"},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.put(
"https://api.gov.nominal.io/api/catalog/v1/datasets/ingest-status-v2",
headers={"Authorization": "Bearer <api key>"},
json={
"status": {"type": "success", "success": {}},
"datasetUuid": "<dataset uuid>",
},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"type": "success",
"success": {}
}
Get ingest progress v2#
GET /catalog/v1/datasets/{datasetRid}/ingest-progress-v2
Deprecated: Deprecated in favor of getDatasetFile.
Path parameters
datasetRidstring (RID) required
Response
startTimestring (date-time)Timestamp at start of ingest
endTimestring (date-time)Timestamp at end of ingest, empty if still in progress
ingestStatusIngestStatusV2Status of ingest, contains error if failed
Set
typeto"success".successSuccessResult
Set
typeto"error".errorErrorResult
Fields of
ErrorResulterrorTypestring
messagestring
errorInstanceIdstring
Set
typeto"inProgress".inProgressInProgressResult
Fields of
InProgressResultinProgressDetailsInProgressDetails
Set
typeto"deletionInProgress".deletionInProgressDeletionInProgress
Set
typeto"deleted".deletedDeleted
Set
typeto"queued".queuedQueued
Set
typeto"parsing".parsingParsing
Set
typeto"ingesting".ingestingIngesting
incalculablebooleanWhether ingest duration can be reliably calculated
Request
curl "https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/ingest-progress-v2" \
-H "Authorization: Bearer <api key>"
import requests
response = requests.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/ingest-progress-v2",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/ingest-progress-v2",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/ingest-progress-v2",
headers={"Authorization": "Bearer <api key>"},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"startTime": "2026-01-01T00:00:00Z",
"endTime": "2026-01-01T00:00:00Z",
"ingestStatus": {
"type": "success",
"success": {}
},
"incalculable": true
}
Get handle for dataset#
GET /catalog/v1/datasets/{dataset}/handle
Deprecated: Returned in Dataset
Path parameters
datasetstring (UUID) required
Response
A Handle union.
Set type to "s3".
s3S3Handle
Fields of
S3Handlebucketstring
keystring
When there is nothing to return, the response is HTTP 204 with no body.
Request
curl "https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset>/handle" \
-H "Authorization: Bearer <api key>"
import requests
response = requests.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset>/handle",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset>/handle",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset>/handle",
headers={"Authorization": "Bearer <api key>"},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
}
Create dataset#
POST /catalog/v1/datasets
Body
namestring required
handleHandledeprecatedDeprecated: Handles should only be specified when adding files to datasets
Set
typeto"s3".s3S3Handle
Fields of
S3Handlebucketstring required
keystring required
metadatamap of string to string deprecatedDeprecated: Deprecated in favor of properties
originMetadataDatasetOriginMetadatarequired
Fields of
DatasetOriginMetadatapathstringDeprecated in favor of FileOriginMetadata
xSeriesIsAbsolutebooleanDeprecated in favor of FileOriginMetadata
schemaDirectivePathstringDeprecated in favor of FileOriginMetadata
xSeriesColumnNamestringDeprecated in favor of FileOriginMetadata
xSeriesTimeUnitTimeUnitDeprecated in favor of FileOriginMetadata
Values of
TimeUnitDAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
timestampMetadataTimestampMetadataDeprecated in favor of FileOriginMetadata
Fields of
TimestampMetadataseriesNamestring required
timestampTypecatalog.TimestampTyperequired
channelConfigChannelConfig
Fields of
ChannelConfigprefixTreeDelimiterstringIf set, will construct a prefix tree for channels of the dataset using the given delimiter.
labelsarray of string
propertiesmap ofPropertyNameto string deprecatedUser-defined dataset properties. The UTF-8 JSON serialization must not exceed 2,000 bytes. Deprecated: use typedProperties
typedPropertiesmap ofPropertyNametoTypedPropertyValue
Set
typeto"stringValue".stringValuestring
Set
typeto"numericValue".numericValuenumber
descriptionstring
granularityGranularityGranularity of dataset timestamps. Defaults to nanoseconds.
Values of
GranularityNANOSECONDS
isV2Datasetboolean deprecatedIf true, the dataset should be ingested to the v2 tables and is compatible with streaming. Deprecated: Regardless of what value is specified, this flag is treated as true by the service.
workspacestring (RID)The workspace in which to create the dataset. If not provided, the dataset will be created in the default workspace for the user’s organization, if the default workspace for the organization is configured.
markingRidsarray of string (RID)The markings to apply to the created dataset. If not provided, the dataset will be visible to all users in the same workspace.
derivedDefinitionCreateDerivedDefinitionDefinition for a derived dataset. When present, the created dataset is virtual and does not ingest or persist dataset files.
Fields of
CreateDerivedDefinitionspecapi.DatasetrequiredInitial derived dataset logic.
Set
typeto"asset".assetAssetSingle asset. Expands to one grouping per data scope on the asset. Groupings are tagged by
assetRidanddataScopeso callers can safely compose multiple assets withcombine.
Set
typeto"run".runRunSingle run. Expands to one grouping per data source attached to the run: asset-typed sources contribute one grouping per data scope on the asset; dataset-typed sources contribute one grouping per dataset. Groupings are tagged by
runRidanddataScope; asset-typed groupings additionally carryassetRidso callers can safely compose multiple runs withcombine.
Set
typeto"saved".savedSavedDataset
Set
typeto"search".searchSearchDataset
Set
typeto"combine".combineCombinedDataset
Set
typeto"tag".tagTaggedDataset
Set
typeto"filter".filterFilteredDataset
Set
typeto"timeShift".timeShiftTimeShiftedDataset
Set
typeto"alignByAnchor".alignByAnchorAnchorAlignedDataset
Set
typeto"withSeries".withSeriesWithSeriesDataset
Set
typeto"dropSeries".dropSeriesDropSeriesDataset
Set
typeto"reference".referenceDatasetReference
Set
typeto"withDimensions".withDimensionsDimensionsDataset
messagestring requiredInitial commit message for the derived definition’s main branch.
channelSearchSplitTagKeysarray of stringTag keys whose distinct values channel search surfaces as separate series. Every key must be produced by
derivedDefinition; ingested tag keys, and keys supplied without aderivedDefinition, are rejected.defaultGroupByAndAggregationConfigDefaultGroupByAndAggregationConfigIf set, the dataset’s default group-by and aggregation config.
Fields of
DefaultGroupByAndAggregationConfignumericAggregationNumericAggregationOperatorIf set, the default aggregation applied to merge points of a numeric series that share an exact timestamp.
Set
typeto"sum".sumSummation
Set
typeto"average".averageAverage
Set
typeto"min".minMinimum
Set
typeto"max".maxMaximum
Set
typeto"count".countCount
Set
typeto"standardDeviation".standardDeviationStandardDeviation
Set
typeto"rootMeanSquare".rootMeanSquareRootMeanSquare
Set
typeto"percentile".percentilePercentile
Set
typeto"udf".
Response
ridstring (RID)
uuidstring (UUID)
namestring
descriptionstring
displayNamestring
metadatamap of string to string deprecatedDeprecated: Deprecated in favor of properties.
handleHandledeprecatedDeprecated: Deprecated. Use DatasetFile#handle
Set
typeto"s3".s3S3Handle
Fields of
S3Handlebucketstring
keystring
ingestDatestring (date-time)Dataset creation time. The name is historical; use lastIngestedAt for when data was last added.
lastIngestedAtstring (date-time)When data was last added to this dataset, as the latest of a successful file ingest and streaming-session activity. Empty when the dataset has never received either.
ingestStatusIngestStatusdeprecatedDeprecated: Deprecated. Use lastIngestStatus.
Values of
IngestStatusIN_PROGRESS
COMPLETED
FAILED
DELETION_IN_PROGRESS
DELETED
originMetadataDatasetOriginMetadata
Fields of
DatasetOriginMetadatapathstringDeprecated in favor of FileOriginMetadata
xSeriesIsAbsolutebooleanDeprecated in favor of FileOriginMetadata
schemaDirectivePathstringDeprecated in favor of FileOriginMetadata
xSeriesColumnNamestringDeprecated in favor of FileOriginMetadata
xSeriesTimeUnitTimeUnitDeprecated in favor of FileOriginMetadata
Values of
TimeUnitDAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
timestampMetadataTimestampMetadataDeprecated in favor of FileOriginMetadata
Fields of
TimestampMetadataseriesNamestring
timestampTypecatalog.TimestampType
channelConfigChannelConfig
Fields of
ChannelConfigprefixTreeDelimiterstringIf set, will construct a prefix tree for channels of the dataset using the given delimiter.
lastIngestStatusIngestStatusV2
Set
typeto"success".successSuccessResult
Set
typeto"error".errorErrorResult
Fields of
ErrorResulterrorTypestring
messagestring
errorInstanceIdstring
Set
typeto"inProgress".inProgressInProgressResult
Fields of
InProgressResultinProgressDetailsInProgressDetails
Set
typeto"deletionInProgress".deletionInProgressDeletionInProgress
Set
typeto"deleted".deletedDeleted
Set
typeto"queued".queuedQueued
Set
typeto"parsing".parsingParsing
Set
typeto"ingesting".ingestingIngesting
Fields of
IngestingprogressnumberOptional fraction between 0 and 1 indicating progress through ingestion.
retentionPolicyRetentionPolicy
Fields of
RetentionPolicytypeRetentionPolicyType
Values of
RetentionPolicyTypeKEEP_UNTIL_EXPIRY
KEEP_FOREVER
expiryDaysinteger
sourcestring
boundsBounds
Fields of
BoundsstartTimestamp
Fields of
Timestampsecondsinteger (int64)
nanosinteger (int64)
endTimestamp
typedatasource.TimestampType
Values of
datasource.TimestampTypeRELATIVE
ABSOLUTE
timestampTypeWeakTimestampType
Values of
WeakTimestampTypeABSOLUTE
RELATIVE
PENDING
labelsarray of string
propertiesmap ofPropertyNameto string deprecatedDeprecated: use typedProperties
typedPropertiesmap ofPropertyNametoTypedPropertyValue
Set
typeto"stringValue".stringValuestring
Set
typeto"numericValue".numericValuenumber
granularityGranularity
Values of
GranularityNANOSECONDS
allowStreamingboolean
isArchivedboolean
defaultGroupByAndAggregationConfigDefaultGroupByAndAggregationConfig
Fields of
DefaultGroupByAndAggregationConfignumericAggregationNumericAggregationOperatorIf set, the default aggregation applied to merge points of a numeric series that share an exact timestamp.
Set
typeto"sum".sumSummation
Set
typeto"average".averageAverage
Set
typeto"min".minMinimum
Set
typeto"max".maxMaximum
Set
typeto"count".countCount
Set
typeto"standardDeviation".standardDeviationStandardDeviation
Set
typeto"rootMeanSquare".rootMeanSquareRootMeanSquare
Set
typeto"percentile".percentilePercentile
Set
typeto"udf".
migratedFromConnectionRidstring (RID)Connection RID whose data source was migrated into this dataset. Empty for datasets that were not created by the legacy connection-to-dataset migrations, or when the original connection row no longer exists.
externalConnectionMetadataExternalConnectionMetadata
Fields of
ExternalConnectionMetadataconnectionDetailsConnectionDetails
Set
typeto"timescale".timescaleTimescaleConnectionDetails
Set
typeto"influx".influxInflux2ConnectionDetails
Set
typeto"influx1".influx1Influx1ConnectionDetails
Set
typeto"nominal".nominalNominalConnectionDetails
Set
typeto"timestream".timestreamTimestreamConnectionDetails
Set
typeto"visualCrossing".visualCrossingVisualCrossingConnectionDetails
Set
typeto"bigQuery".bigQueryBigQueryConnectionDetails
connectionStatusConnectionStatus
Fields of
ConnectionStatusisConnectedboolean
limitsLimitsConfig
Fields of
LimitsConfigmaxQueryRangeconnection.api.Duration
derivedDefinitionDerivedDefinitionDefinition for a derived dataset. When present, this dataset is virtual and backed by the referenced provided Dataset definition instead of persisted dataset files.
Fields of
DerivedDefinitionspecapi.Dataset
Set
typeto"asset".assetAssetSingle asset. Expands to one grouping per data scope on the asset. Groupings are tagged by
assetRidanddataScopeso callers can safely compose multiple assets withcombine.
Set
typeto"run".runRunSingle run. Expands to one grouping per data source attached to the run: asset-typed sources contribute one grouping per data scope on the asset; dataset-typed sources contribute one grouping per dataset. Groupings are tagged by
runRidanddataScope; asset-typed groupings additionally carryassetRidso callers can safely compose multiple runs withcombine.
Set
typeto"saved".savedSavedDataset
Set
typeto"search".searchSearchDataset
Set
typeto"combine".combineCombinedDataset
Set
typeto"tag".tagTaggedDataset
Set
typeto"filter".filterFilteredDataset
Set
typeto"timeShift".timeShiftTimeShiftedDataset
Set
typeto"alignByAnchor".alignByAnchorAnchorAlignedDataset
Set
typeto"withSeries".withSeriesWithSeriesDataset
Set
typeto"dropSeries".dropSeriesDropSeriesDataset
Set
typeto"reference".referenceDatasetReference
Set
typeto"withDimensions".withDimensionsDimensionsDataset
commitCommit
Fields of
Commitidstring
resourceRidstring (RID)
parentCommitstring
mergeParentCommitstring
messagestring
isWorkingStatebooleanA working state commit is created via the
saveWorkingStateendpoint and is non-permanent. In the future, it may be compacted and not exist.committedBystring (RID)
committedAtstring (date-time)
channelSearchSplitTagKeysarray of stringTag keys whose distinct values channel search surfaces as separate series. Each listed key is retained on this dataset’s channel-search locators, so a channel carrying two values of that key appears once per value instead of collapsing into a single entry — the behavior a data scope has on an asset. Keys not listed here stay available for filtering and grouping but do not contribute to series cardinality. Every key must be produced by this dataset’s
derivedDefinition. Ingested tag keys cannot be split on.
Request
curl -X POST "https://api.gov.nominal.io/api/catalog/v1/datasets" \
-H "Authorization: Bearer <api key>" \
-H "Content-Type: application/json" \
--data '{
"name": "<name>",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": true,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS"
}
}
},
"channelConfig": {
"prefixTreeDelimiter": "<prefix tree delimiter>"
}
}
}'
import requests
response = requests.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets",
headers={"Authorization": "Bearer <api key>"},
json={
"name": "<name>",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": True,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {"type": "relative", "relative": {"timeUnit": "DAYS"}},
},
"channelConfig": {"prefixTreeDelimiter": "<prefix tree delimiter>"},
},
},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets",
headers={"Authorization": "Bearer <api key>"},
json={
"name": "<name>",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": True,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {"type": "relative", "relative": {"timeUnit": "DAYS"}},
},
"channelConfig": {"prefixTreeDelimiter": "<prefix tree delimiter>"},
},
},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets",
headers={"Authorization": "Bearer <api key>"},
json={
"name": "<name>",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": True,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {"timeUnit": "DAYS"},
},
},
"channelConfig": {"prefixTreeDelimiter": "<prefix tree delimiter>"},
},
},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"rid": "<rid>",
"uuid": "<uuid>",
"name": "<name>",
"description": "<description>",
"displayName": "<display name>",
"metadata": {
"<key>": "<value>"
},
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"ingestDate": "2026-01-01T00:00:00Z",
"lastIngestedAt": "2026-01-01T00:00:00Z",
"ingestStatus": "IN_PROGRESS",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": true,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS",
"offset": "2026-01-01T00:00:00Z"
}
}
},
"channelConfig": {
"prefixTreeDelimiter": "<prefix tree delimiter>"
}
},
"lastIngestStatus": {
"type": "success",
"success": {}
},
"retentionPolicy": {
"type": "KEEP_UNTIL_EXPIRY",
"expiryDays": 1
},
"source": "<source>",
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"timestampType": "ABSOLUTE",
"labels": [
"<label>"
],
"properties": {
"<key>": "<property value>"
},
"typedProperties": {
"<key>": {
"type": "stringValue",
"stringValue": "<property value>"
}
},
"granularity": "NANOSECONDS",
"allowStreaming": true,
"isArchived": true,
"defaultGroupByAndAggregationConfig": {
"numericAggregation": {
"type": "sum",
"sum": {}
}
},
"migratedFromConnectionRid": "<connection rid>",
"externalConnectionMetadata": {
"connectionDetails": {
"type": "timescale",
"timescale": {
"database": "<database>",
"host": "<host>",
"port": 1,
"username": "<username>",
"passwordSecretRid": "<secret rid>"
}
},
"connectionStatus": {
"isConnected": true
},
"limits": {
"maxQueryRange": {
"unit": "HOURS",
"count": 1
}
}
},
"derivedDefinition": {
"spec": {
"type": "asset",
"asset": {
"rid": {
"type": "literal",
"literal": "<literal>"
}
}
},
"commit": {
"id": "<commit id>",
"resourceRid": "<resource rid>",
"parentCommit": "<commit id>",
"mergeParentCommit": "<commit id>",
"message": "<message>",
"isWorkingState": true,
"committedBy": "<user rid>",
"committedAt": "2026-01-01T00:00:00Z"
}
},
"channelSearchSplitTagKeys": [
"<tag name>"
]
}
Create or update dataset#
POST /catalog/v1/datasets/create-or-update
Deprecated: Use createDataset instead.
Creates a dataset if the s3 path does not exist, otherwise updates the dataset
Body
namestring required
handleHandledeprecatedDeprecated: Handles should only be specified when adding files to datasets
Set
typeto"s3".s3S3Handle
Fields of
S3Handlebucketstring required
keystring required
metadatamap of string to string deprecatedDeprecated: Deprecated in favor of properties
originMetadataDatasetOriginMetadatarequired
Fields of
DatasetOriginMetadatapathstringDeprecated in favor of FileOriginMetadata
xSeriesIsAbsolutebooleanDeprecated in favor of FileOriginMetadata
schemaDirectivePathstringDeprecated in favor of FileOriginMetadata
xSeriesColumnNamestringDeprecated in favor of FileOriginMetadata
xSeriesTimeUnitTimeUnitDeprecated in favor of FileOriginMetadata
Values of
TimeUnitDAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
timestampMetadataTimestampMetadataDeprecated in favor of FileOriginMetadata
Fields of
TimestampMetadataseriesNamestring required
timestampTypecatalog.TimestampTyperequired
channelConfigChannelConfig
Fields of
ChannelConfigprefixTreeDelimiterstringIf set, will construct a prefix tree for channels of the dataset using the given delimiter.
labelsarray of string
propertiesmap ofPropertyNameto string deprecatedUser-defined dataset properties. The UTF-8 JSON serialization must not exceed 2,000 bytes. Deprecated: use typedProperties
typedPropertiesmap ofPropertyNametoTypedPropertyValue
Set
typeto"stringValue".stringValuestring
Set
typeto"numericValue".numericValuenumber
descriptionstring
granularityGranularityGranularity of dataset timestamps. Defaults to nanoseconds.
Values of
GranularityNANOSECONDS
isV2Datasetboolean deprecatedIf true, the dataset should be ingested to the v2 tables and is compatible with streaming. Deprecated: Regardless of what value is specified, this flag is treated as true by the service.
workspacestring (RID)The workspace in which to create the dataset. If not provided, the dataset will be created in the default workspace for the user’s organization, if the default workspace for the organization is configured.
markingRidsarray of string (RID)The markings to apply to the created dataset. If not provided, the dataset will be visible to all users in the same workspace.
derivedDefinitionCreateDerivedDefinitionDefinition for a derived dataset. When present, the created dataset is virtual and does not ingest or persist dataset files.
Fields of
CreateDerivedDefinitionspecapi.DatasetrequiredInitial derived dataset logic.
Set
typeto"asset".assetAssetSingle asset. Expands to one grouping per data scope on the asset. Groupings are tagged by
assetRidanddataScopeso callers can safely compose multiple assets withcombine.
Set
typeto"run".runRunSingle run. Expands to one grouping per data source attached to the run: asset-typed sources contribute one grouping per data scope on the asset; dataset-typed sources contribute one grouping per dataset. Groupings are tagged by
runRidanddataScope; asset-typed groupings additionally carryassetRidso callers can safely compose multiple runs withcombine.
Set
typeto"saved".savedSavedDataset
Set
typeto"search".searchSearchDataset
Set
typeto"combine".combineCombinedDataset
Set
typeto"tag".tagTaggedDataset
Set
typeto"filter".filterFilteredDataset
Set
typeto"timeShift".timeShiftTimeShiftedDataset
Set
typeto"alignByAnchor".alignByAnchorAnchorAlignedDataset
Set
typeto"withSeries".withSeriesWithSeriesDataset
Set
typeto"dropSeries".dropSeriesDropSeriesDataset
Set
typeto"reference".referenceDatasetReference
Set
typeto"withDimensions".withDimensionsDimensionsDataset
messagestring requiredInitial commit message for the derived definition’s main branch.
channelSearchSplitTagKeysarray of stringTag keys whose distinct values channel search surfaces as separate series. Every key must be produced by
derivedDefinition; ingested tag keys, and keys supplied without aderivedDefinition, are rejected.defaultGroupByAndAggregationConfigDefaultGroupByAndAggregationConfigIf set, the dataset’s default group-by and aggregation config.
Fields of
DefaultGroupByAndAggregationConfignumericAggregationNumericAggregationOperatorIf set, the default aggregation applied to merge points of a numeric series that share an exact timestamp.
Set
typeto"sum".sumSummation
Set
typeto"average".averageAverage
Set
typeto"min".minMinimum
Set
typeto"max".maxMaximum
Set
typeto"count".countCount
Set
typeto"standardDeviation".standardDeviationStandardDeviation
Set
typeto"rootMeanSquare".rootMeanSquareRootMeanSquare
Set
typeto"percentile".percentilePercentile
Set
typeto"udf".
Response
ridstring (RID)
uuidstring (UUID)
namestring
descriptionstring
displayNamestring
metadatamap of string to string deprecatedDeprecated: Deprecated in favor of properties.
handleHandledeprecatedDeprecated: Deprecated. Use DatasetFile#handle
Set
typeto"s3".s3S3Handle
Fields of
S3Handlebucketstring
keystring
ingestDatestring (date-time)Dataset creation time. The name is historical; use lastIngestedAt for when data was last added.
lastIngestedAtstring (date-time)When data was last added to this dataset, as the latest of a successful file ingest and streaming-session activity. Empty when the dataset has never received either.
ingestStatusIngestStatusdeprecatedDeprecated: Deprecated. Use lastIngestStatus.
Values of
IngestStatusIN_PROGRESS
COMPLETED
FAILED
DELETION_IN_PROGRESS
DELETED
originMetadataDatasetOriginMetadata
Fields of
DatasetOriginMetadatapathstringDeprecated in favor of FileOriginMetadata
xSeriesIsAbsolutebooleanDeprecated in favor of FileOriginMetadata
schemaDirectivePathstringDeprecated in favor of FileOriginMetadata
xSeriesColumnNamestringDeprecated in favor of FileOriginMetadata
xSeriesTimeUnitTimeUnitDeprecated in favor of FileOriginMetadata
Values of
TimeUnitDAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
timestampMetadataTimestampMetadataDeprecated in favor of FileOriginMetadata
Fields of
TimestampMetadataseriesNamestring
timestampTypecatalog.TimestampType
channelConfigChannelConfig
Fields of
ChannelConfigprefixTreeDelimiterstringIf set, will construct a prefix tree for channels of the dataset using the given delimiter.
lastIngestStatusIngestStatusV2
Set
typeto"success".successSuccessResult
Set
typeto"error".errorErrorResult
Fields of
ErrorResulterrorTypestring
messagestring
errorInstanceIdstring
Set
typeto"inProgress".inProgressInProgressResult
Fields of
InProgressResultinProgressDetailsInProgressDetails
Set
typeto"deletionInProgress".deletionInProgressDeletionInProgress
Set
typeto"deleted".deletedDeleted
Set
typeto"queued".queuedQueued
Set
typeto"parsing".parsingParsing
Set
typeto"ingesting".ingestingIngesting
Fields of
IngestingprogressnumberOptional fraction between 0 and 1 indicating progress through ingestion.
retentionPolicyRetentionPolicy
Fields of
RetentionPolicytypeRetentionPolicyType
Values of
RetentionPolicyTypeKEEP_UNTIL_EXPIRY
KEEP_FOREVER
expiryDaysinteger
sourcestring
boundsBounds
Fields of
BoundsstartTimestamp
Fields of
Timestampsecondsinteger (int64)
nanosinteger (int64)
endTimestamp
typedatasource.TimestampType
Values of
datasource.TimestampTypeRELATIVE
ABSOLUTE
timestampTypeWeakTimestampType
Values of
WeakTimestampTypeABSOLUTE
RELATIVE
PENDING
labelsarray of string
propertiesmap ofPropertyNameto string deprecatedDeprecated: use typedProperties
typedPropertiesmap ofPropertyNametoTypedPropertyValue
Set
typeto"stringValue".stringValuestring
Set
typeto"numericValue".numericValuenumber
granularityGranularity
Values of
GranularityNANOSECONDS
allowStreamingboolean
isArchivedboolean
defaultGroupByAndAggregationConfigDefaultGroupByAndAggregationConfig
Fields of
DefaultGroupByAndAggregationConfignumericAggregationNumericAggregationOperatorIf set, the default aggregation applied to merge points of a numeric series that share an exact timestamp.
Set
typeto"sum".sumSummation
Set
typeto"average".averageAverage
Set
typeto"min".minMinimum
Set
typeto"max".maxMaximum
Set
typeto"count".countCount
Set
typeto"standardDeviation".standardDeviationStandardDeviation
Set
typeto"rootMeanSquare".rootMeanSquareRootMeanSquare
Set
typeto"percentile".percentilePercentile
Set
typeto"udf".
migratedFromConnectionRidstring (RID)Connection RID whose data source was migrated into this dataset. Empty for datasets that were not created by the legacy connection-to-dataset migrations, or when the original connection row no longer exists.
externalConnectionMetadataExternalConnectionMetadata
Fields of
ExternalConnectionMetadataconnectionDetailsConnectionDetails
Set
typeto"timescale".timescaleTimescaleConnectionDetails
Set
typeto"influx".influxInflux2ConnectionDetails
Set
typeto"influx1".influx1Influx1ConnectionDetails
Set
typeto"nominal".nominalNominalConnectionDetails
Set
typeto"timestream".timestreamTimestreamConnectionDetails
Set
typeto"visualCrossing".visualCrossingVisualCrossingConnectionDetails
Set
typeto"bigQuery".bigQueryBigQueryConnectionDetails
connectionStatusConnectionStatus
Fields of
ConnectionStatusisConnectedboolean
limitsLimitsConfig
Fields of
LimitsConfigmaxQueryRangeconnection.api.Duration
derivedDefinitionDerivedDefinitionDefinition for a derived dataset. When present, this dataset is virtual and backed by the referenced provided Dataset definition instead of persisted dataset files.
Fields of
DerivedDefinitionspecapi.Dataset
Set
typeto"asset".assetAssetSingle asset. Expands to one grouping per data scope on the asset. Groupings are tagged by
assetRidanddataScopeso callers can safely compose multiple assets withcombine.
Set
typeto"run".runRunSingle run. Expands to one grouping per data source attached to the run: asset-typed sources contribute one grouping per data scope on the asset; dataset-typed sources contribute one grouping per dataset. Groupings are tagged by
runRidanddataScope; asset-typed groupings additionally carryassetRidso callers can safely compose multiple runs withcombine.
Set
typeto"saved".savedSavedDataset
Set
typeto"search".searchSearchDataset
Set
typeto"combine".combineCombinedDataset
Set
typeto"tag".tagTaggedDataset
Set
typeto"filter".filterFilteredDataset
Set
typeto"timeShift".timeShiftTimeShiftedDataset
Set
typeto"alignByAnchor".alignByAnchorAnchorAlignedDataset
Set
typeto"withSeries".withSeriesWithSeriesDataset
Set
typeto"dropSeries".dropSeriesDropSeriesDataset
Set
typeto"reference".referenceDatasetReference
Set
typeto"withDimensions".withDimensionsDimensionsDataset
commitCommit
Fields of
Commitidstring
resourceRidstring (RID)
parentCommitstring
mergeParentCommitstring
messagestring
isWorkingStatebooleanA working state commit is created via the
saveWorkingStateendpoint and is non-permanent. In the future, it may be compacted and not exist.committedBystring (RID)
committedAtstring (date-time)
channelSearchSplitTagKeysarray of stringTag keys whose distinct values channel search surfaces as separate series. Each listed key is retained on this dataset’s channel-search locators, so a channel carrying two values of that key appears once per value instead of collapsing into a single entry — the behavior a data scope has on an asset. Keys not listed here stay available for filtering and grouping but do not contribute to series cardinality. Every key must be produced by this dataset’s
derivedDefinition. Ingested tag keys cannot be split on.
Request
curl -X POST "https://api.gov.nominal.io/api/catalog/v1/datasets/create-or-update" \
-H "Authorization: Bearer <api key>" \
-H "Content-Type: application/json" \
--data '{
"name": "<name>",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": true,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS"
}
}
},
"channelConfig": {
"prefixTreeDelimiter": "<prefix tree delimiter>"
}
}
}'
import requests
response = requests.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/create-or-update",
headers={"Authorization": "Bearer <api key>"},
json={
"name": "<name>",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": True,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {"type": "relative", "relative": {"timeUnit": "DAYS"}},
},
"channelConfig": {"prefixTreeDelimiter": "<prefix tree delimiter>"},
},
},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/create-or-update",
headers={"Authorization": "Bearer <api key>"},
json={
"name": "<name>",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": True,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {"type": "relative", "relative": {"timeUnit": "DAYS"}},
},
"channelConfig": {"prefixTreeDelimiter": "<prefix tree delimiter>"},
},
},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/create-or-update",
headers={"Authorization": "Bearer <api key>"},
json={
"name": "<name>",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": True,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {"timeUnit": "DAYS"},
},
},
"channelConfig": {"prefixTreeDelimiter": "<prefix tree delimiter>"},
},
},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"rid": "<rid>",
"uuid": "<uuid>",
"name": "<name>",
"description": "<description>",
"displayName": "<display name>",
"metadata": {
"<key>": "<value>"
},
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"ingestDate": "2026-01-01T00:00:00Z",
"lastIngestedAt": "2026-01-01T00:00:00Z",
"ingestStatus": "IN_PROGRESS",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": true,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS",
"offset": "2026-01-01T00:00:00Z"
}
}
},
"channelConfig": {
"prefixTreeDelimiter": "<prefix tree delimiter>"
}
},
"lastIngestStatus": {
"type": "success",
"success": {}
},
"retentionPolicy": {
"type": "KEEP_UNTIL_EXPIRY",
"expiryDays": 1
},
"source": "<source>",
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"timestampType": "ABSOLUTE",
"labels": [
"<label>"
],
"properties": {
"<key>": "<property value>"
},
"typedProperties": {
"<key>": {
"type": "stringValue",
"stringValue": "<property value>"
}
},
"granularity": "NANOSECONDS",
"allowStreaming": true,
"isArchived": true,
"defaultGroupByAndAggregationConfig": {
"numericAggregation": {
"type": "sum",
"sum": {}
}
},
"migratedFromConnectionRid": "<connection rid>",
"externalConnectionMetadata": {
"connectionDetails": {
"type": "timescale",
"timescale": {
"database": "<database>",
"host": "<host>",
"port": 1,
"username": "<username>",
"passwordSecretRid": "<secret rid>"
}
},
"connectionStatus": {
"isConnected": true
},
"limits": {
"maxQueryRange": {
"unit": "HOURS",
"count": 1
}
}
},
"derivedDefinition": {
"spec": {
"type": "asset",
"asset": {
"rid": {
"type": "literal",
"literal": "<literal>"
}
}
},
"commit": {
"id": "<commit id>",
"resourceRid": "<resource rid>",
"parentCommit": "<commit id>",
"mergeParentCommit": "<commit id>",
"message": "<message>",
"isWorkingState": true,
"committedBy": "<user rid>",
"committedAt": "2026-01-01T00:00:00Z"
}
},
"channelSearchSplitTagKeys": [
"<tag name>"
]
}
Create dataset with uuid#
POST /catalog/v1/datasets/with-uuid
Creates a dataset with a specific UUID. This is useful for migrations and advanced use cases where the dataset UUID must be controlled by the caller. Throws a conflict error if a dataset with the specified UUID already exists. This endpoint is not intended for general use. Use /datasets instead to create a new dataset.
Body
uuidstring (UUID) requiredThe UUID to assign to the new dataset.
createDatasetCreateDatasetrequired
Fields of
CreateDatasetnamestring required
handleHandledeprecatedDeprecated: Handles should only be specified when adding files to datasets
Set
typeto"s3".s3S3Handle
metadatamap of string to string deprecatedDeprecated: Deprecated in favor of properties
originMetadataDatasetOriginMetadatarequired
Fields of
DatasetOriginMetadatapathstringDeprecated in favor of FileOriginMetadata
xSeriesIsAbsolutebooleanDeprecated in favor of FileOriginMetadata
schemaDirectivePathstringDeprecated in favor of FileOriginMetadata
xSeriesColumnNamestringDeprecated in favor of FileOriginMetadata
xSeriesTimeUnitTimeUnitDeprecated in favor of FileOriginMetadata
timestampMetadataTimestampMetadataDeprecated in favor of FileOriginMetadata
channelConfigChannelConfig
labelsarray of string
propertiesmap ofPropertyNameto string deprecatedUser-defined dataset properties. The UTF-8 JSON serialization must not exceed 2,000 bytes. Deprecated: use typedProperties
typedPropertiesmap ofPropertyNametoTypedPropertyValue
Set
typeto"stringValue".stringValuestring
Set
typeto"numericValue".numericValuenumber
descriptionstring
granularityGranularityGranularity of dataset timestamps. Defaults to nanoseconds.
Values of
GranularityNANOSECONDS
isV2Datasetboolean deprecatedIf true, the dataset should be ingested to the v2 tables and is compatible with streaming. Deprecated: Regardless of what value is specified, this flag is treated as true by the service.
workspacestring (RID)The workspace in which to create the dataset. If not provided, the dataset will be created in the default workspace for the user’s organization, if the default workspace for the organization is configured.
markingRidsarray of string (RID)The markings to apply to the created dataset. If not provided, the dataset will be visible to all users in the same workspace.
derivedDefinitionCreateDerivedDefinitionDefinition for a derived dataset. When present, the created dataset is virtual and does not ingest or persist dataset files.
Fields of
CreateDerivedDefinitionspecapi.DatasetrequiredInitial derived dataset logic.
messagestring requiredInitial commit message for the derived definition’s main branch.
channelSearchSplitTagKeysarray of stringTag keys whose distinct values channel search surfaces as separate series. Every key must be produced by
derivedDefinition; ingested tag keys, and keys supplied without aderivedDefinition, are rejected.defaultGroupByAndAggregationConfigDefaultGroupByAndAggregationConfigIf set, the dataset’s default group-by and aggregation config.
Fields of
DefaultGroupByAndAggregationConfignumericAggregationNumericAggregationOperatorIf set, the default aggregation applied to merge points of a numeric series that share an exact timestamp.
Response
ridstring (RID)
uuidstring (UUID)
namestring
descriptionstring
displayNamestring
metadatamap of string to string deprecatedDeprecated: Deprecated in favor of properties.
handleHandledeprecatedDeprecated: Deprecated. Use DatasetFile#handle
Set
typeto"s3".s3S3Handle
Fields of
S3Handlebucketstring
keystring
ingestDatestring (date-time)Dataset creation time. The name is historical; use lastIngestedAt for when data was last added.
lastIngestedAtstring (date-time)When data was last added to this dataset, as the latest of a successful file ingest and streaming-session activity. Empty when the dataset has never received either.
ingestStatusIngestStatusdeprecatedDeprecated: Deprecated. Use lastIngestStatus.
Values of
IngestStatusIN_PROGRESS
COMPLETED
FAILED
DELETION_IN_PROGRESS
DELETED
originMetadataDatasetOriginMetadata
Fields of
DatasetOriginMetadatapathstringDeprecated in favor of FileOriginMetadata
xSeriesIsAbsolutebooleanDeprecated in favor of FileOriginMetadata
schemaDirectivePathstringDeprecated in favor of FileOriginMetadata
xSeriesColumnNamestringDeprecated in favor of FileOriginMetadata
xSeriesTimeUnitTimeUnitDeprecated in favor of FileOriginMetadata
Values of
TimeUnitDAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
timestampMetadataTimestampMetadataDeprecated in favor of FileOriginMetadata
Fields of
TimestampMetadataseriesNamestring
timestampTypecatalog.TimestampType
channelConfigChannelConfig
Fields of
ChannelConfigprefixTreeDelimiterstringIf set, will construct a prefix tree for channels of the dataset using the given delimiter.
lastIngestStatusIngestStatusV2
Set
typeto"success".successSuccessResult
Set
typeto"error".errorErrorResult
Fields of
ErrorResulterrorTypestring
messagestring
errorInstanceIdstring
Set
typeto"inProgress".inProgressInProgressResult
Fields of
InProgressResultinProgressDetailsInProgressDetails
Set
typeto"deletionInProgress".deletionInProgressDeletionInProgress
Set
typeto"deleted".deletedDeleted
Set
typeto"queued".queuedQueued
Set
typeto"parsing".parsingParsing
Set
typeto"ingesting".ingestingIngesting
Fields of
IngestingprogressnumberOptional fraction between 0 and 1 indicating progress through ingestion.
retentionPolicyRetentionPolicy
Fields of
RetentionPolicytypeRetentionPolicyType
Values of
RetentionPolicyTypeKEEP_UNTIL_EXPIRY
KEEP_FOREVER
expiryDaysinteger
sourcestring
boundsBounds
Fields of
BoundsstartTimestamp
Fields of
Timestampsecondsinteger (int64)
nanosinteger (int64)
endTimestamp
typedatasource.TimestampType
Values of
datasource.TimestampTypeRELATIVE
ABSOLUTE
timestampTypeWeakTimestampType
Values of
WeakTimestampTypeABSOLUTE
RELATIVE
PENDING
labelsarray of string
propertiesmap ofPropertyNameto string deprecatedDeprecated: use typedProperties
typedPropertiesmap ofPropertyNametoTypedPropertyValue
Set
typeto"stringValue".stringValuestring
Set
typeto"numericValue".numericValuenumber
granularityGranularity
Values of
GranularityNANOSECONDS
allowStreamingboolean
isArchivedboolean
defaultGroupByAndAggregationConfigDefaultGroupByAndAggregationConfig
Fields of
DefaultGroupByAndAggregationConfignumericAggregationNumericAggregationOperatorIf set, the default aggregation applied to merge points of a numeric series that share an exact timestamp.
Set
typeto"sum".sumSummation
Set
typeto"average".averageAverage
Set
typeto"min".minMinimum
Set
typeto"max".maxMaximum
Set
typeto"count".countCount
Set
typeto"standardDeviation".standardDeviationStandardDeviation
Set
typeto"rootMeanSquare".rootMeanSquareRootMeanSquare
Set
typeto"percentile".percentilePercentile
Set
typeto"udf".
migratedFromConnectionRidstring (RID)Connection RID whose data source was migrated into this dataset. Empty for datasets that were not created by the legacy connection-to-dataset migrations, or when the original connection row no longer exists.
externalConnectionMetadataExternalConnectionMetadata
Fields of
ExternalConnectionMetadataconnectionDetailsConnectionDetails
Set
typeto"timescale".timescaleTimescaleConnectionDetails
Set
typeto"influx".influxInflux2ConnectionDetails
Set
typeto"influx1".influx1Influx1ConnectionDetails
Set
typeto"nominal".nominalNominalConnectionDetails
Set
typeto"timestream".timestreamTimestreamConnectionDetails
Set
typeto"visualCrossing".visualCrossingVisualCrossingConnectionDetails
Set
typeto"bigQuery".bigQueryBigQueryConnectionDetails
connectionStatusConnectionStatus
Fields of
ConnectionStatusisConnectedboolean
limitsLimitsConfig
Fields of
LimitsConfigmaxQueryRangeconnection.api.Duration
derivedDefinitionDerivedDefinitionDefinition for a derived dataset. When present, this dataset is virtual and backed by the referenced provided Dataset definition instead of persisted dataset files.
Fields of
DerivedDefinitionspecapi.Dataset
Set
typeto"asset".assetAssetSingle asset. Expands to one grouping per data scope on the asset. Groupings are tagged by
assetRidanddataScopeso callers can safely compose multiple assets withcombine.
Set
typeto"run".runRunSingle run. Expands to one grouping per data source attached to the run: asset-typed sources contribute one grouping per data scope on the asset; dataset-typed sources contribute one grouping per dataset. Groupings are tagged by
runRidanddataScope; asset-typed groupings additionally carryassetRidso callers can safely compose multiple runs withcombine.
Set
typeto"saved".savedSavedDataset
Set
typeto"search".searchSearchDataset
Set
typeto"combine".combineCombinedDataset
Set
typeto"tag".tagTaggedDataset
Set
typeto"filter".filterFilteredDataset
Set
typeto"timeShift".timeShiftTimeShiftedDataset
Set
typeto"alignByAnchor".alignByAnchorAnchorAlignedDataset
Set
typeto"withSeries".withSeriesWithSeriesDataset
Set
typeto"dropSeries".dropSeriesDropSeriesDataset
Set
typeto"reference".referenceDatasetReference
Set
typeto"withDimensions".withDimensionsDimensionsDataset
commitCommit
Fields of
Commitidstring
resourceRidstring (RID)
parentCommitstring
mergeParentCommitstring
messagestring
isWorkingStatebooleanA working state commit is created via the
saveWorkingStateendpoint and is non-permanent. In the future, it may be compacted and not exist.committedBystring (RID)
committedAtstring (date-time)
channelSearchSplitTagKeysarray of stringTag keys whose distinct values channel search surfaces as separate series. Each listed key is retained on this dataset’s channel-search locators, so a channel carrying two values of that key appears once per value instead of collapsing into a single entry — the behavior a data scope has on an asset. Keys not listed here stay available for filtering and grouping but do not contribute to series cardinality. Every key must be produced by this dataset’s
derivedDefinition. Ingested tag keys cannot be split on.
Request
curl -X POST "https://api.gov.nominal.io/api/catalog/v1/datasets/with-uuid" \
-H "Authorization: Bearer <api key>" \
-H "Content-Type: application/json" \
--data '{
"uuid": "<uuid>",
"createDataset": {
"name": "<name>",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": true,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS"
}
}
},
"channelConfig": {
"prefixTreeDelimiter": "<prefix tree delimiter>"
}
}
}
}'
import requests
response = requests.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/with-uuid",
headers={"Authorization": "Bearer <api key>"},
json={
"uuid": "<uuid>",
"createDataset": {
"name": "<name>",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": True,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {"timeUnit": "DAYS"},
},
},
"channelConfig": {"prefixTreeDelimiter": "<prefix tree delimiter>"},
},
},
},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/with-uuid",
headers={"Authorization": "Bearer <api key>"},
json={
"uuid": "<uuid>",
"createDataset": {
"name": "<name>",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": True,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {"timeUnit": "DAYS"},
},
},
"channelConfig": {"prefixTreeDelimiter": "<prefix tree delimiter>"},
},
},
},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/with-uuid",
headers={"Authorization": "Bearer <api key>"},
json={
"uuid": "<uuid>",
"createDataset": {
"name": "<name>",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": True,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {"timeUnit": "DAYS"},
},
},
"channelConfig": {
"prefixTreeDelimiter": "<prefix tree delimiter>",
},
},
},
},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"rid": "<rid>",
"uuid": "<uuid>",
"name": "<name>",
"description": "<description>",
"displayName": "<display name>",
"metadata": {
"<key>": "<value>"
},
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"ingestDate": "2026-01-01T00:00:00Z",
"lastIngestedAt": "2026-01-01T00:00:00Z",
"ingestStatus": "IN_PROGRESS",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": true,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS",
"offset": "2026-01-01T00:00:00Z"
}
}
},
"channelConfig": {
"prefixTreeDelimiter": "<prefix tree delimiter>"
}
},
"lastIngestStatus": {
"type": "success",
"success": {}
},
"retentionPolicy": {
"type": "KEEP_UNTIL_EXPIRY",
"expiryDays": 1
},
"source": "<source>",
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"timestampType": "ABSOLUTE",
"labels": [
"<label>"
],
"properties": {
"<key>": "<property value>"
},
"typedProperties": {
"<key>": {
"type": "stringValue",
"stringValue": "<property value>"
}
},
"granularity": "NANOSECONDS",
"allowStreaming": true,
"isArchived": true,
"defaultGroupByAndAggregationConfig": {
"numericAggregation": {
"type": "sum",
"sum": {}
}
},
"migratedFromConnectionRid": "<connection rid>",
"externalConnectionMetadata": {
"connectionDetails": {
"type": "timescale",
"timescale": {
"database": "<database>",
"host": "<host>",
"port": 1,
"username": "<username>",
"passwordSecretRid": "<secret rid>"
}
},
"connectionStatus": {
"isConnected": true
},
"limits": {
"maxQueryRange": {
"unit": "HOURS",
"count": 1
}
}
},
"derivedDefinition": {
"spec": {
"type": "asset",
"asset": {
"rid": {
"type": "literal",
"literal": "<literal>"
}
}
},
"commit": {
"id": "<commit id>",
"resourceRid": "<resource rid>",
"parentCommit": "<commit id>",
"mergeParentCommit": "<commit id>",
"message": "<message>",
"isWorkingState": true,
"committedBy": "<user rid>",
"committedAt": "2026-01-01T00:00:00Z"
}
},
"channelSearchSplitTagKeys": [
"<tag name>"
]
}
Add file to dataset#
POST /catalog/v1/datasets/{datasetRid}/add-file
Adds a single file to an existing dataset.
Path parameters
datasetRidstring (RID) required
Body
handleHandlerequired
Set
typeto"s3".s3S3Handle
Fields of
S3Handlebucketstring required
keystring required
fileSizeinteger (int64) requiredThe size of the file in bytes.
timestampMetadataTimestampMetadata
Fields of
TimestampMetadataseriesNamestring required
timestampTypecatalog.TimestampTyperequired
Set
typeto"relative".relativeRelativeTimestamp
Fields of
RelativeTimestamptimeUnitTimeUnitrequired
Values of
TimeUnitDAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
offsetstring (date-time)
Set
typeto"absolute".absoluteAbsoluteTimestamp
Set
typeto"iso8601".iso8601Iso8601Timestamp
Set
typeto"epochOfTimeUnit".epochOfTimeUnitEpochTimestamp
Fields of
EpochTimestamptimeUnitTimeUnitrequired
Set
typeto"customFormat".customFormatCustomTimestamp
Fields of
CustomTimestampformatstring requiredThe format string should be in the format of the
DateTimeFormatterclass in Java.defaultYearintegerYear is accepted as an optional field for cases like IRIG time format, and will be assumed as current year if not provided.
defaultDayOfYearintegerDefault day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.
ingestTagMetadataIngestTagMetadata
Fields of
IngestTagMetadatatagColumnsmap ofTagNameto stringA map of tag names to column names to derive the tag values from.
additionalFileTagsmap ofTagNameto string
originFileHandlesarray ofS3Handle
ingestJobRidstring (RID)
metadataDatasetFileMetadataFile-type-specific metadata. For video files, contains timestamp manifest.
Set
typeto"video".videoVideoFileMetadata
Fields of
VideoFileMetadatatimestampManifestVideoFileTimestampManifestrequiredSpecifies how to determine absolute timestamps for each frame in the video. Can be an embedded MCAP manifest, an external sidecar file, or calculated from a starting offset applied to presentation timestamps.
Set
typeto"mcap".mcapMcapTimestampManifest
Fields of
McapTimestampManifestmcapChannelLocatorMcapChannelLocatorrequired
Set
typeto"topic".topicstring
Set
typeto"id".idinteger
Set
typeto"s3path".s3pathstring
Set
typeto"s3paths".s3pathsarray of stringMultiple sidecar manifest files whose per-frame timestamps are concatenated in list order. Used when a video’s frame timestamps are recorded across several files.
Set
typeto"noManifest".noManifestNoTimestampManifest
Fields of
NoTimestampManifeststartingTimestampTimestamprequiredSpecifies the original starting timestamp of the video.
Fields of
Timestampsecondsinteger (int64) required
nanosinteger (int64) required
scaleParameterScaleParameterA field that specifies that the frame rate of the video does not match the frame rate of the camera | i.e. a slowed down or sped up video. Can specify either the camera frame rate or the absolute end time.
Set
typeto"trueFrameRate".trueFrameRatenumber
Set
typeto"endingTimestamp".endingTimestampTimestamp
Set
typeto"scaleFactor".scaleFactornumberthe scale factor can be used to calculate whether media duration differs from a video’s | real duration, and if so, the true frame rate of the camera. The video time will thus be scaled | by the ratio of the real duration to media duration, or media frame rate to true frame rate.
segmentMetadataVideoSegmentsMetadataCached aggregate metadata about the segments comprising this video file after segmentation has completed. Includes frame counts, duration, and frame rate. Empty until segmentation is complete. Note: Min/max timestamps are stored in DatasetFile.bounds field. Raw file size is stored in DatasetFile.fileSizeBytes.
Fields of
VideoSegmentsMetadatanumFramesinteger requiredTotal number of frames across all segments.
numSegmentsinteger requiredNumber of segments the video was split into.
scaleFactornumber requiredScale factor applied to timestamps during segmentation.
mediaDurationSecondsnumber requiredTotal duration of the video in seconds.
mediaFrameRatenumber requiredAverage frame rate (FPS) calculated as total frames / duration.
Response
idstring (UUID)
datasetRidstring (RID)
namestring
handleHandle
Set
typeto"s3".s3S3Handle
Fields of
S3Handlebucketstring
keystring
fileSizeBytesinteger (int64)Size in bytes of the stored dataset file; for transforming ingests this is the produced output, not the original upload.
boundsBounds
Fields of
BoundsstartTimestamp
Fields of
Timestampsecondsinteger (int64)
nanosinteger (int64)
endTimestamp
typedatasource.TimestampType
Values of
datasource.TimestampTypeRELATIVE
ABSOLUTE
uploadedAtstring (date-time)Timestamp that the file was received and stored, but not processed or made available to consumers.
ingestedAtstring (date-time)Timestamp that the file is ingested at and made available for processing. If the file has failed to be ingested for any reason or is still being processed, then this value will be empty.
ingestStatusIngestStatusV2Ingest status. The in-progress sub-phases (queued, parsing, ingesting) are surfaced as inProgress with the phase carried in InProgressResult.inProgressDetails, so consumers see a stable top-level lifecycle.
Set
typeto"success".successSuccessResult
Set
typeto"error".errorErrorResult
Fields of
ErrorResulterrorTypestring
messagestring
errorInstanceIdstring
Set
typeto"inProgress".inProgressInProgressResult
Fields of
InProgressResultinProgressDetailsInProgressDetails
Set
typeto"deletionInProgress".deletionInProgressDeletionInProgress
Set
typeto"deleted".deletedDeleted
Set
typeto"queued".queuedQueued
Set
typeto"parsing".parsingParsing
Set
typeto"ingesting".ingestingIngesting
timestampMetadataTimestampMetadata
Fields of
TimestampMetadataseriesNamestring
timestampTypecatalog.TimestampType
Set
typeto"relative".relativeRelativeTimestamp
Fields of
RelativeTimestamptimeUnitTimeUnit
Values of
TimeUnitDAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
offsetstring (date-time)
Set
typeto"absolute".absoluteAbsoluteTimestamp
Set
typeto"iso8601".iso8601Iso8601Timestamp
Set
typeto"epochOfTimeUnit".epochOfTimeUnitEpochTimestamp
Fields of
EpochTimestamptimeUnitTimeUnit
Set
typeto"customFormat".customFormatCustomTimestamp
Fields of
CustomTimestampformatstringThe format string should be in the format of the
DateTimeFormatterclass in Java.defaultYearintegerYear is accepted as an optional field for cases like IRIG time format, and will be assumed as current year if not provided.
defaultDayOfYearintegerDefault day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.
ingestTagMetadataIngestTagMetadata
Fields of
IngestTagMetadatatagColumnsmap ofTagNameto stringA map of tag names to column names to derive the tag values from.
additionalFileTagsmap ofTagNameto string
originFilePathsarray of string
ingestJobRidstring (RID)
ingestTransformRidstring (RID)The ingest transform within ingestJobRid that produced this file. An ingest job may fan out to many transforms, so this pins the file to the specific unit of work (and extractor) that created it — resolvable via IngestJobService#getIngestTransform. Extractor logs are tagged with this transform’s UUID (the RID locator), so this also links a file to the logs of the transform that produced it. Absent for directly uploaded files and for files ingested before this link was recorded.
deletedAtstring (date-time)Timestamp that the file is deleted at, only present if the file has been deleted.
metadataDatasetFileMetadataFile-type-specific metadata. For video files, contains timestamp manifest and segment metadata.
Set
typeto"video".videoVideoFileMetadata
Fields of
VideoFileMetadatatimestampManifestVideoFileTimestampManifestSpecifies how to determine absolute timestamps for each frame in the video. Can be an embedded MCAP manifest, an external sidecar file, or calculated from a starting offset applied to presentation timestamps.
Set
typeto"mcap".mcapMcapTimestampManifest
Fields of
McapTimestampManifestmcapChannelLocatorMcapChannelLocator
Set
typeto"topic".topicstring
Set
typeto"id".idinteger
Set
typeto"s3path".s3pathstring
Set
typeto"s3paths".s3pathsarray of stringMultiple sidecar manifest files whose per-frame timestamps are concatenated in list order. Used when a video’s frame timestamps are recorded across several files.
Set
typeto"noManifest".noManifestNoTimestampManifest
Fields of
NoTimestampManifeststartingTimestampTimestampSpecifies the original starting timestamp of the video.
scaleParameterScaleParameterA field that specifies that the frame rate of the video does not match the frame rate of the camera | i.e. a slowed down or sped up video. Can specify either the camera frame rate or the absolute end time.
Set
typeto"trueFrameRate".trueFrameRatenumber
Set
typeto"endingTimestamp".endingTimestampTimestamp
Set
typeto"scaleFactor".scaleFactornumberthe scale factor can be used to calculate whether media duration differs from a video’s | real duration, and if so, the true frame rate of the camera. The video time will thus be scaled | by the ratio of the real duration to media duration, or media frame rate to true frame rate.
segmentMetadataVideoSegmentsMetadataCached aggregate metadata about the segments comprising this video file after segmentation has completed. Includes frame counts, duration, and frame rate. Empty until segmentation is complete. Note: Min/max timestamps are stored in DatasetFile.bounds field. Raw file size is stored in DatasetFile.fileSizeBytes.
Fields of
VideoSegmentsMetadatanumFramesintegerTotal number of frames across all segments.
numSegmentsintegerNumber of segments the video was split into.
scaleFactornumberScale factor applied to timestamps during segmentation.
mediaDurationSecondsnumberTotal duration of the video in seconds.
mediaFrameRatenumberAverage frame rate (FPS) calculated as total frames / duration.
Request
curl -X POST "https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/add-file" \
-H "Authorization: Bearer <api key>" \
-H "Content-Type: application/json" \
--data '{
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"fileSize": 1
}'
import requests
response = requests.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/add-file",
headers={"Authorization": "Bearer <api key>"},
json={
"handle": {"type": "s3", "s3": {"bucket": "<bucket>", "key": "<key>"}},
"fileSize": 1,
},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/add-file",
headers={"Authorization": "Bearer <api key>"},
json={
"handle": {"type": "s3", "s3": {"bucket": "<bucket>", "key": "<key>"}},
"fileSize": 1,
},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/add-file",
headers={"Authorization": "Bearer <api key>"},
json={
"handle": {"type": "s3", "s3": {"bucket": "<bucket>", "key": "<key>"}},
"fileSize": 1,
},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"id": "<dataset file id>",
"datasetRid": "<dataset rid>",
"name": "<name>",
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"fileSizeBytes": 1,
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"uploadedAt": "2026-01-01T00:00:00Z",
"ingestedAt": "2026-01-01T00:00:00Z",
"ingestStatus": {
"type": "success",
"success": {}
},
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS",
"offset": "2026-01-01T00:00:00Z"
}
}
},
"ingestTagMetadata": {
"tagColumns": {
"<key>": "<column name>"
},
"additionalFileTags": {
"<key>": "<tag value>"
}
},
"originFilePaths": [
"<origin file paths>"
],
"ingestJobRid": "<ingest job rid>",
"ingestTransformRid": "<ingest transform rid>",
"deletedAt": "2026-01-01T00:00:00Z",
"metadata": {
"type": "video",
"video": {
"timestampManifest": {
"type": "mcap",
"mcap": {
"mcapChannelLocator": {
"type": "topic",
"topic": "<mcap channel topic>"
}
}
},
"segmentMetadata": {
"numFrames": 1,
"numSegments": 1,
"scaleFactor": 1,
"mediaDurationSeconds": 1,
"mediaFrameRate": 1
}
}
}
}
List dataset files#
GET /catalog/v1/datasets/{datasetRid}/files
Path parameters
datasetRidstring (RID) required
Query parameters
nextPageTokenstring
Response
filesarray ofDatasetFile
Fields of
DatasetFileidstring (UUID)
datasetRidstring (RID)
namestring
handleHandle
Set
typeto"s3".s3S3Handle
Fields of
S3Handlebucketstring
keystring
fileSizeBytesinteger (int64)Size in bytes of the stored dataset file; for transforming ingests this is the produced output, not the original upload.
boundsBounds
Fields of
BoundsstartTimestamp
Fields of
Timestampsecondsinteger (int64)
nanosinteger (int64)
endTimestamp
typedatasource.TimestampType
Values of
datasource.TimestampTypeRELATIVE
ABSOLUTE
uploadedAtstring (date-time)Timestamp that the file was received and stored, but not processed or made available to consumers.
ingestedAtstring (date-time)Timestamp that the file is ingested at and made available for processing. If the file has failed to be ingested for any reason or is still being processed, then this value will be empty.
ingestStatusIngestStatusV2Ingest status. The in-progress sub-phases (queued, parsing, ingesting) are surfaced as inProgress with the phase carried in InProgressResult.inProgressDetails, so consumers see a stable top-level lifecycle.
Set
typeto"success".successSuccessResult
Set
typeto"error".errorErrorResult
Fields of
ErrorResulterrorTypestring
messagestring
errorInstanceIdstring
Set
typeto"inProgress".inProgressInProgressResult
Fields of
InProgressResultinProgressDetailsInProgressDetails
Set
typeto"deletionInProgress".deletionInProgressDeletionInProgress
Set
typeto"deleted".deletedDeleted
Set
typeto"queued".queuedQueued
Set
typeto"parsing".parsingParsing
Set
typeto"ingesting".ingestingIngesting
timestampMetadataTimestampMetadata
Fields of
TimestampMetadataseriesNamestring
timestampTypecatalog.TimestampType
Set
typeto"relative".relativeRelativeTimestamp
Fields of
RelativeTimestamptimeUnitTimeUnit
Values of
TimeUnitDAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
offsetstring (date-time)
Set
typeto"absolute".absoluteAbsoluteTimestamp
Set
typeto"iso8601".iso8601Iso8601Timestamp
Set
typeto"epochOfTimeUnit".epochOfTimeUnitEpochTimestamp
Fields of
EpochTimestamptimeUnitTimeUnit
Set
typeto"customFormat".customFormatCustomTimestamp
Fields of
CustomTimestampformatstringThe format string should be in the format of the
DateTimeFormatterclass in Java.defaultYearintegerYear is accepted as an optional field for cases like IRIG time format, and will be assumed as current year if not provided.
defaultDayOfYearintegerDefault day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.
ingestTagMetadataIngestTagMetadata
Fields of
IngestTagMetadatatagColumnsmap ofTagNameto stringA map of tag names to column names to derive the tag values from.
additionalFileTagsmap ofTagNameto string
originFilePathsarray of string
ingestJobRidstring (RID)
ingestTransformRidstring (RID)The ingest transform within ingestJobRid that produced this file. An ingest job may fan out to many transforms, so this pins the file to the specific unit of work (and extractor) that created it — resolvable via IngestJobService#getIngestTransform. Extractor logs are tagged with this transform’s UUID (the RID locator), so this also links a file to the logs of the transform that produced it. Absent for directly uploaded files and for files ingested before this link was recorded.
deletedAtstring (date-time)Timestamp that the file is deleted at, only present if the file has been deleted.
metadataDatasetFileMetadataFile-type-specific metadata. For video files, contains timestamp manifest and segment metadata.
Set
typeto"video".videoVideoFileMetadata
Fields of
VideoFileMetadatatimestampManifestVideoFileTimestampManifestSpecifies how to determine absolute timestamps for each frame in the video. Can be an embedded MCAP manifest, an external sidecar file, or calculated from a starting offset applied to presentation timestamps.
Set
typeto"mcap".mcapMcapTimestampManifest
Fields of
McapTimestampManifestmcapChannelLocatorMcapChannelLocator
Set
typeto"topic".topicstring
Set
typeto"id".idinteger
Set
typeto"s3path".s3pathstring
Set
typeto"s3paths".s3pathsarray of stringMultiple sidecar manifest files whose per-frame timestamps are concatenated in list order. Used when a video’s frame timestamps are recorded across several files.
Set
typeto"noManifest".noManifestNoTimestampManifest
Fields of
NoTimestampManifeststartingTimestampTimestampSpecifies the original starting timestamp of the video.
scaleParameterScaleParameterA field that specifies that the frame rate of the video does not match the frame rate of the camera | i.e. a slowed down or sped up video. Can specify either the camera frame rate or the absolute end time.
Set
typeto"trueFrameRate".trueFrameRatenumber
Set
typeto"endingTimestamp".endingTimestampTimestamp
Set
typeto"scaleFactor".scaleFactornumberthe scale factor can be used to calculate whether media duration differs from a video’s | real duration, and if so, the true frame rate of the camera. The video time will thus be scaled | by the ratio of the real duration to media duration, or media frame rate to true frame rate.
segmentMetadataVideoSegmentsMetadataCached aggregate metadata about the segments comprising this video file after segmentation has completed. Includes frame counts, duration, and frame rate. Empty until segmentation is complete. Note: Min/max timestamps are stored in DatasetFile.bounds field. Raw file size is stored in DatasetFile.fileSizeBytes.
Fields of
VideoSegmentsMetadatanumFramesintegerTotal number of frames across all segments.
numSegmentsintegerNumber of segments the video was split into.
scaleFactornumberScale factor applied to timestamps during segmentation.
mediaDurationSecondsnumberTotal duration of the video in seconds.
mediaFrameRatenumberAverage frame rate (FPS) calculated as total frames / duration.
nextPagestring
Request
curl "https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/files" \
-H "Authorization: Bearer <api key>"
import requests
response = requests.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/files",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/files",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/files",
headers={"Authorization": "Bearer <api key>"},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"files": [
{
"id": "<dataset file id>",
"datasetRid": "<dataset rid>",
"name": "<name>",
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"fileSizeBytes": 1,
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"uploadedAt": "2026-01-01T00:00:00Z",
"ingestedAt": "2026-01-01T00:00:00Z",
"ingestStatus": {
"type": "success",
"success": {}
},
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS",
"offset": "2026-01-01T00:00:00Z"
}
}
},
"ingestTagMetadata": {
"tagColumns": {
"<key>": "<column name>"
},
"additionalFileTags": {
"<key>": "<tag value>"
}
},
"originFilePaths": [
"<origin file paths>"
],
"ingestJobRid": "<ingest job rid>",
"ingestTransformRid": "<ingest transform rid>",
"deletedAt": "2026-01-01T00:00:00Z",
"metadata": {
"type": "video",
"video": {
"timestampManifest": {
"type": "mcap",
"mcap": {
"mcapChannelLocator": {
"type": "topic",
"topic": "<mcap channel topic>"
}
}
},
"segmentMetadata": {
"numFrames": 1,
"numSegments": 1,
"scaleFactor": 1,
"mediaDurationSeconds": 1,
"mediaFrameRate": 1
}
}
}
}
],
"nextPage": "<token>"
}
Search dataset files#
POST /catalog/v1/search-dataset-files
Body
datasetRidstring (RID) required
querySearchDatasetFilesQueryrequired
Set
typeto"timeRange".timeRangeTimeRangeFilter
Fields of
TimeRangeFilterstartUtcTimestamp
Fields of
UtcTimestampsecondsSinceEpochinteger (int64) required
endUtcTimestamp
Set
typeto"uploadedAtRange".uploadedAtRangeUploadedAtFilter
Fields of
UploadedAtFilterstartUtcTimestamp
endUtcTimestamp
Set
typeto"fileTags".fileTagsmap ofTagNameto string
Set
typeto"and".andarray ofSearchDatasetFilesQuery
Set
typeto"or".orarray ofSearchDatasetFilesQuery
pageSizeintegerDefaults to 100. Will throw if larger than 1000.
tokenstring
sortOptionsDatasetFileSortOptionsrequired
Fields of
DatasetFileSortOptionsisDescendingboolean required
fieldDatasetFileSortFieldrequired
Values of
DatasetFileSortFieldUPLOADED_AT
Response
resultsarray ofDatasetFile
Fields of
DatasetFileidstring (UUID)
datasetRidstring (RID)
namestring
handleHandle
Set
typeto"s3".s3S3Handle
Fields of
S3Handlebucketstring
keystring
fileSizeBytesinteger (int64)Size in bytes of the stored dataset file; for transforming ingests this is the produced output, not the original upload.
boundsBounds
Fields of
BoundsstartTimestamp
Fields of
Timestampsecondsinteger (int64)
nanosinteger (int64)
endTimestamp
typedatasource.TimestampType
Values of
datasource.TimestampTypeRELATIVE
ABSOLUTE
uploadedAtstring (date-time)Timestamp that the file was received and stored, but not processed or made available to consumers.
ingestedAtstring (date-time)Timestamp that the file is ingested at and made available for processing. If the file has failed to be ingested for any reason or is still being processed, then this value will be empty.
ingestStatusIngestStatusV2Ingest status. The in-progress sub-phases (queued, parsing, ingesting) are surfaced as inProgress with the phase carried in InProgressResult.inProgressDetails, so consumers see a stable top-level lifecycle.
Set
typeto"success".successSuccessResult
Set
typeto"error".errorErrorResult
Fields of
ErrorResulterrorTypestring
messagestring
errorInstanceIdstring
Set
typeto"inProgress".inProgressInProgressResult
Fields of
InProgressResultinProgressDetailsInProgressDetails
Set
typeto"deletionInProgress".deletionInProgressDeletionInProgress
Set
typeto"deleted".deletedDeleted
Set
typeto"queued".queuedQueued
Set
typeto"parsing".parsingParsing
Set
typeto"ingesting".ingestingIngesting
timestampMetadataTimestampMetadata
Fields of
TimestampMetadataseriesNamestring
timestampTypecatalog.TimestampType
Set
typeto"relative".relativeRelativeTimestamp
Fields of
RelativeTimestamptimeUnitTimeUnit
Values of
TimeUnitDAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
offsetstring (date-time)
Set
typeto"absolute".absoluteAbsoluteTimestamp
Set
typeto"iso8601".iso8601Iso8601Timestamp
Set
typeto"epochOfTimeUnit".epochOfTimeUnitEpochTimestamp
Fields of
EpochTimestamptimeUnitTimeUnit
Set
typeto"customFormat".customFormatCustomTimestamp
Fields of
CustomTimestampformatstringThe format string should be in the format of the
DateTimeFormatterclass in Java.defaultYearintegerYear is accepted as an optional field for cases like IRIG time format, and will be assumed as current year if not provided.
defaultDayOfYearintegerDefault day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.
ingestTagMetadataIngestTagMetadata
Fields of
IngestTagMetadatatagColumnsmap ofTagNameto stringA map of tag names to column names to derive the tag values from.
additionalFileTagsmap ofTagNameto string
originFilePathsarray of string
ingestJobRidstring (RID)
ingestTransformRidstring (RID)The ingest transform within ingestJobRid that produced this file. An ingest job may fan out to many transforms, so this pins the file to the specific unit of work (and extractor) that created it — resolvable via IngestJobService#getIngestTransform. Extractor logs are tagged with this transform’s UUID (the RID locator), so this also links a file to the logs of the transform that produced it. Absent for directly uploaded files and for files ingested before this link was recorded.
deletedAtstring (date-time)Timestamp that the file is deleted at, only present if the file has been deleted.
metadataDatasetFileMetadataFile-type-specific metadata. For video files, contains timestamp manifest and segment metadata.
Set
typeto"video".videoVideoFileMetadata
Fields of
VideoFileMetadatatimestampManifestVideoFileTimestampManifestSpecifies how to determine absolute timestamps for each frame in the video. Can be an embedded MCAP manifest, an external sidecar file, or calculated from a starting offset applied to presentation timestamps.
Set
typeto"mcap".mcapMcapTimestampManifest
Fields of
McapTimestampManifestmcapChannelLocatorMcapChannelLocator
Set
typeto"topic".topicstring
Set
typeto"id".idinteger
Set
typeto"s3path".s3pathstring
Set
typeto"s3paths".s3pathsarray of stringMultiple sidecar manifest files whose per-frame timestamps are concatenated in list order. Used when a video’s frame timestamps are recorded across several files.
Set
typeto"noManifest".noManifestNoTimestampManifest
Fields of
NoTimestampManifeststartingTimestampTimestampSpecifies the original starting timestamp of the video.
scaleParameterScaleParameterA field that specifies that the frame rate of the video does not match the frame rate of the camera | i.e. a slowed down or sped up video. Can specify either the camera frame rate or the absolute end time.
Set
typeto"trueFrameRate".trueFrameRatenumber
Set
typeto"endingTimestamp".endingTimestampTimestamp
Set
typeto"scaleFactor".scaleFactornumberthe scale factor can be used to calculate whether media duration differs from a video’s | real duration, and if so, the true frame rate of the camera. The video time will thus be scaled | by the ratio of the real duration to media duration, or media frame rate to true frame rate.
segmentMetadataVideoSegmentsMetadataCached aggregate metadata about the segments comprising this video file after segmentation has completed. Includes frame counts, duration, and frame rate. Empty until segmentation is complete. Note: Min/max timestamps are stored in DatasetFile.bounds field. Raw file size is stored in DatasetFile.fileSizeBytes.
Fields of
VideoSegmentsMetadatanumFramesintegerTotal number of frames across all segments.
numSegmentsintegerNumber of segments the video was split into.
scaleFactornumberScale factor applied to timestamps during segmentation.
mediaDurationSecondsnumberTotal duration of the video in seconds.
mediaFrameRatenumberAverage frame rate (FPS) calculated as total frames / duration.
nextPageTokenstring
Request
curl -X POST "https://api.gov.nominal.io/api/catalog/v1/search-dataset-files" \
-H "Authorization: Bearer <api key>" \
-H "Content-Type: application/json" \
--data '{
"datasetRid": "<dataset rid>",
"query": {
"type": "timeRange",
"timeRange": {
"start": {
"secondsSinceEpoch": 1
},
"end": {
"secondsSinceEpoch": 1
}
}
},
"sortOptions": {
"isDescending": true,
"field": "UPLOADED_AT"
}
}'
import requests
response = requests.post(
"https://api.gov.nominal.io/api/catalog/v1/search-dataset-files",
headers={"Authorization": "Bearer <api key>"},
json={
"datasetRid": "<dataset rid>",
"query": {
"type": "timeRange",
"timeRange": {
"start": {"secondsSinceEpoch": 1},
"end": {"secondsSinceEpoch": 1},
},
},
"sortOptions": {"isDescending": True, "field": "UPLOADED_AT"},
},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.post(
"https://api.gov.nominal.io/api/catalog/v1/search-dataset-files",
headers={"Authorization": "Bearer <api key>"},
json={
"datasetRid": "<dataset rid>",
"query": {
"type": "timeRange",
"timeRange": {
"start": {"secondsSinceEpoch": 1},
"end": {"secondsSinceEpoch": 1},
},
},
"sortOptions": {"isDescending": True, "field": "UPLOADED_AT"},
},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.post(
"https://api.gov.nominal.io/api/catalog/v1/search-dataset-files",
headers={"Authorization": "Bearer <api key>"},
json={
"datasetRid": "<dataset rid>",
"query": {
"type": "timeRange",
"timeRange": {
"start": {"secondsSinceEpoch": 1},
"end": {"secondsSinceEpoch": 1},
},
},
"sortOptions": {"isDescending": True, "field": "UPLOADED_AT"},
},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"results": [
{
"id": "<dataset file id>",
"datasetRid": "<dataset rid>",
"name": "<name>",
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"fileSizeBytes": 1,
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"uploadedAt": "2026-01-01T00:00:00Z",
"ingestedAt": "2026-01-01T00:00:00Z",
"ingestStatus": {
"type": "success",
"success": {}
},
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS",
"offset": "2026-01-01T00:00:00Z"
}
}
},
"ingestTagMetadata": {
"tagColumns": {
"<key>": "<column name>"
},
"additionalFileTags": {
"<key>": "<tag value>"
}
},
"originFilePaths": [
"<origin file paths>"
],
"ingestJobRid": "<ingest job rid>",
"ingestTransformRid": "<ingest transform rid>",
"deletedAt": "2026-01-01T00:00:00Z",
"metadata": {
"type": "video",
"video": {
"timestampManifest": {
"type": "mcap",
"mcap": {
"mcapChannelLocator": {
"type": "topic",
"topic": "<mcap channel topic>"
}
}
},
"segmentMetadata": {
"numFrames": 1,
"numSegments": 1,
"scaleFactor": 1,
"mediaDurationSeconds": 1,
"mediaFrameRate": 1
}
}
}
}
],
"nextPageToken": "<token>"
}
Get dataset file uri#
GET /catalog/v1/datasets/{datasetRid}/{fileId}/uri
Path parameters
datasetRidstring (RID) required
fileIdstring (UUID) required
Response
uristring
Request
curl "https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/<file id>/uri" \
-H "Authorization: Bearer <api key>"
import requests
response = requests.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/<file id>/uri",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/<file id>/uri",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/<file id>/uri",
headers={"Authorization": "Bearer <api key>"},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"uri": "<uri>"
}
Get video file uri#
GET /catalog/v1/videos/{videoFileRid}/uri
Path parameters
videoFileRidstring (RID) required
Response
uristring
Request
curl "https://api.gov.nominal.io/api/catalog/v1/videos/<video file rid>/uri" \
-H "Authorization: Bearer <api key>"
import requests
response = requests.get(
"https://api.gov.nominal.io/api/catalog/v1/videos/<video file rid>/uri",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.get(
"https://api.gov.nominal.io/api/catalog/v1/videos/<video file rid>/uri",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.get(
"https://api.gov.nominal.io/api/catalog/v1/videos/<video file rid>/uri",
headers={"Authorization": "Bearer <api key>"},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"uri": "<uri>"
}
Get origin file uris#
GET /catalog/v1/datasets/{datasetRid}/{fileId}/origin-uris
Path parameters
datasetRidstring (RID) required
fileIdstring (UUID) required
Response
The body is an array of OriginFileUri.
Fields of OriginFileUri
uristring
pathstring
Request
curl "https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/<file id>/origin-uris" \
-H "Authorization: Bearer <api key>"
import requests
response = requests.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/<file id>/origin-uris",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/<file id>/origin-uris",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/<file id>/origin-uris",
headers={"Authorization": "Bearer <api key>"},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
[
{
"uri": "<uri>",
"path": "<path>"
}
]
Mark file ingest successful#
PUT /catalog/v1/datasets/{datasetRid}/file/{fileId}/success
Path parameters
datasetRidstring (RID) required
fileIdstring (UUID) required
Body
boundsBoundsrequired
Fields of
BoundsstartTimestamprequired
Fields of
Timestampsecondsinteger (int64) required
nanosinteger (int64) required
endTimestamprequired
typedatasource.TimestampTyperequired
Values of
datasource.TimestampTypeRELATIVE
ABSOLUTE
ingestedAtTimestamprequiredThe ingestion timestamp is produced by CSV splitter and stored directly in the data store table. It’s produced externally and passed here to handle retries and failures, and must be nanosecond precision. Two files cannot have the same ingested at timestamp.
Response
idstring (UUID)
datasetRidstring (RID)
namestring
handleHandle
Set
typeto"s3".s3S3Handle
Fields of
S3Handlebucketstring
keystring
fileSizeBytesinteger (int64)Size in bytes of the stored dataset file; for transforming ingests this is the produced output, not the original upload.
boundsBounds
Fields of
BoundsstartTimestamp
Fields of
Timestampsecondsinteger (int64)
nanosinteger (int64)
endTimestamp
typedatasource.TimestampType
Values of
datasource.TimestampTypeRELATIVE
ABSOLUTE
uploadedAtstring (date-time)Timestamp that the file was received and stored, but not processed or made available to consumers.
ingestedAtstring (date-time)Timestamp that the file is ingested at and made available for processing. If the file has failed to be ingested for any reason or is still being processed, then this value will be empty.
ingestStatusIngestStatusV2Ingest status. The in-progress sub-phases (queued, parsing, ingesting) are surfaced as inProgress with the phase carried in InProgressResult.inProgressDetails, so consumers see a stable top-level lifecycle.
Set
typeto"success".successSuccessResult
Set
typeto"error".errorErrorResult
Fields of
ErrorResulterrorTypestring
messagestring
errorInstanceIdstring
Set
typeto"inProgress".inProgressInProgressResult
Fields of
InProgressResultinProgressDetailsInProgressDetails
Set
typeto"deletionInProgress".deletionInProgressDeletionInProgress
Set
typeto"deleted".deletedDeleted
Set
typeto"queued".queuedQueued
Set
typeto"parsing".parsingParsing
Set
typeto"ingesting".ingestingIngesting
timestampMetadataTimestampMetadata
Fields of
TimestampMetadataseriesNamestring
timestampTypecatalog.TimestampType
Set
typeto"relative".relativeRelativeTimestamp
Fields of
RelativeTimestamptimeUnitTimeUnit
Values of
TimeUnitDAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
offsetstring (date-time)
Set
typeto"absolute".absoluteAbsoluteTimestamp
Set
typeto"iso8601".iso8601Iso8601Timestamp
Set
typeto"epochOfTimeUnit".epochOfTimeUnitEpochTimestamp
Fields of
EpochTimestamptimeUnitTimeUnit
Set
typeto"customFormat".customFormatCustomTimestamp
Fields of
CustomTimestampformatstringThe format string should be in the format of the
DateTimeFormatterclass in Java.defaultYearintegerYear is accepted as an optional field for cases like IRIG time format, and will be assumed as current year if not provided.
defaultDayOfYearintegerDefault day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.
ingestTagMetadataIngestTagMetadata
Fields of
IngestTagMetadatatagColumnsmap ofTagNameto stringA map of tag names to column names to derive the tag values from.
additionalFileTagsmap ofTagNameto string
originFilePathsarray of string
ingestJobRidstring (RID)
ingestTransformRidstring (RID)The ingest transform within ingestJobRid that produced this file. An ingest job may fan out to many transforms, so this pins the file to the specific unit of work (and extractor) that created it — resolvable via IngestJobService#getIngestTransform. Extractor logs are tagged with this transform’s UUID (the RID locator), so this also links a file to the logs of the transform that produced it. Absent for directly uploaded files and for files ingested before this link was recorded.
deletedAtstring (date-time)Timestamp that the file is deleted at, only present if the file has been deleted.
metadataDatasetFileMetadataFile-type-specific metadata. For video files, contains timestamp manifest and segment metadata.
Set
typeto"video".videoVideoFileMetadata
Fields of
VideoFileMetadatatimestampManifestVideoFileTimestampManifestSpecifies how to determine absolute timestamps for each frame in the video. Can be an embedded MCAP manifest, an external sidecar file, or calculated from a starting offset applied to presentation timestamps.
Set
typeto"mcap".mcapMcapTimestampManifest
Fields of
McapTimestampManifestmcapChannelLocatorMcapChannelLocator
Set
typeto"topic".topicstring
Set
typeto"id".idinteger
Set
typeto"s3path".s3pathstring
Set
typeto"s3paths".s3pathsarray of stringMultiple sidecar manifest files whose per-frame timestamps are concatenated in list order. Used when a video’s frame timestamps are recorded across several files.
Set
typeto"noManifest".noManifestNoTimestampManifest
Fields of
NoTimestampManifeststartingTimestampTimestampSpecifies the original starting timestamp of the video.
scaleParameterScaleParameterA field that specifies that the frame rate of the video does not match the frame rate of the camera | i.e. a slowed down or sped up video. Can specify either the camera frame rate or the absolute end time.
Set
typeto"trueFrameRate".trueFrameRatenumber
Set
typeto"endingTimestamp".endingTimestampTimestamp
Set
typeto"scaleFactor".scaleFactornumberthe scale factor can be used to calculate whether media duration differs from a video’s | real duration, and if so, the true frame rate of the camera. The video time will thus be scaled | by the ratio of the real duration to media duration, or media frame rate to true frame rate.
segmentMetadataVideoSegmentsMetadataCached aggregate metadata about the segments comprising this video file after segmentation has completed. Includes frame counts, duration, and frame rate. Empty until segmentation is complete. Note: Min/max timestamps are stored in DatasetFile.bounds field. Raw file size is stored in DatasetFile.fileSizeBytes.
Fields of
VideoSegmentsMetadatanumFramesintegerTotal number of frames across all segments.
numSegmentsintegerNumber of segments the video was split into.
scaleFactornumberScale factor applied to timestamps during segmentation.
mediaDurationSecondsnumberTotal duration of the video in seconds.
mediaFrameRatenumberAverage frame rate (FPS) calculated as total frames / duration.
Request
curl -X PUT "https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/file/<file id>/success" \
-H "Authorization: Bearer <api key>" \
-H "Content-Type: application/json" \
--data '{
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"ingestedAt": {
"seconds": 1,
"nanos": 1
}
}'
import requests
response = requests.put(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/file/<file id>/success",
headers={"Authorization": "Bearer <api key>"},
json={
"bounds": {
"start": {"seconds": 1, "nanos": 1},
"end": {"seconds": 1, "nanos": 1},
"type": "RELATIVE",
},
"ingestedAt": {"seconds": 1, "nanos": 1},
},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.put(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/file/<file id>/success",
headers={"Authorization": "Bearer <api key>"},
json={
"bounds": {
"start": {"seconds": 1, "nanos": 1},
"end": {"seconds": 1, "nanos": 1},
"type": "RELATIVE",
},
"ingestedAt": {"seconds": 1, "nanos": 1},
},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.put(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/file/<file id>/success",
headers={"Authorization": "Bearer <api key>"},
json={
"bounds": {
"start": {"seconds": 1, "nanos": 1},
"end": {"seconds": 1, "nanos": 1},
"type": "RELATIVE",
},
"ingestedAt": {"seconds": 1, "nanos": 1},
},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"id": "<dataset file id>",
"datasetRid": "<dataset rid>",
"name": "<name>",
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"fileSizeBytes": 1,
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"uploadedAt": "2026-01-01T00:00:00Z",
"ingestedAt": "2026-01-01T00:00:00Z",
"ingestStatus": {
"type": "success",
"success": {}
},
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS",
"offset": "2026-01-01T00:00:00Z"
}
}
},
"ingestTagMetadata": {
"tagColumns": {
"<key>": "<column name>"
},
"additionalFileTags": {
"<key>": "<tag value>"
}
},
"originFilePaths": [
"<origin file paths>"
],
"ingestJobRid": "<ingest job rid>",
"ingestTransformRid": "<ingest transform rid>",
"deletedAt": "2026-01-01T00:00:00Z",
"metadata": {
"type": "video",
"video": {
"timestampManifest": {
"type": "mcap",
"mcap": {
"mcapChannelLocator": {
"type": "topic",
"topic": "<mcap channel topic>"
}
}
},
"segmentMetadata": {
"numFrames": 1,
"numSegments": 1,
"scaleFactor": 1,
"mediaDurationSeconds": 1,
"mediaFrameRate": 1
}
}
}
}
Mark file ingest error#
PUT /catalog/v1/datasets/{datasetRid}/file/{fileId}/error
Path parameters
datasetRidstring (RID) required
fileIdstring (UUID) required
Body
errorResultErrorResultrequired
Fields of
ErrorResulterrorTypestring required
messagestring required
errorInstanceIdstring
Response
idstring (UUID)
datasetRidstring (RID)
namestring
handleHandle
Set
typeto"s3".s3S3Handle
Fields of
S3Handlebucketstring
keystring
fileSizeBytesinteger (int64)Size in bytes of the stored dataset file; for transforming ingests this is the produced output, not the original upload.
boundsBounds
Fields of
BoundsstartTimestamp
Fields of
Timestampsecondsinteger (int64)
nanosinteger (int64)
endTimestamp
typedatasource.TimestampType
Values of
datasource.TimestampTypeRELATIVE
ABSOLUTE
uploadedAtstring (date-time)Timestamp that the file was received and stored, but not processed or made available to consumers.
ingestedAtstring (date-time)Timestamp that the file is ingested at and made available for processing. If the file has failed to be ingested for any reason or is still being processed, then this value will be empty.
ingestStatusIngestStatusV2Ingest status. The in-progress sub-phases (queued, parsing, ingesting) are surfaced as inProgress with the phase carried in InProgressResult.inProgressDetails, so consumers see a stable top-level lifecycle.
Set
typeto"success".successSuccessResult
Set
typeto"error".errorErrorResult
Fields of
ErrorResulterrorTypestring
messagestring
errorInstanceIdstring
Set
typeto"inProgress".inProgressInProgressResult
Fields of
InProgressResultinProgressDetailsInProgressDetails
Set
typeto"deletionInProgress".deletionInProgressDeletionInProgress
Set
typeto"deleted".deletedDeleted
Set
typeto"queued".queuedQueued
Set
typeto"parsing".parsingParsing
Set
typeto"ingesting".ingestingIngesting
timestampMetadataTimestampMetadata
Fields of
TimestampMetadataseriesNamestring
timestampTypecatalog.TimestampType
Set
typeto"relative".relativeRelativeTimestamp
Fields of
RelativeTimestamptimeUnitTimeUnit
Values of
TimeUnitDAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
offsetstring (date-time)
Set
typeto"absolute".absoluteAbsoluteTimestamp
Set
typeto"iso8601".iso8601Iso8601Timestamp
Set
typeto"epochOfTimeUnit".epochOfTimeUnitEpochTimestamp
Fields of
EpochTimestamptimeUnitTimeUnit
Set
typeto"customFormat".customFormatCustomTimestamp
Fields of
CustomTimestampformatstringThe format string should be in the format of the
DateTimeFormatterclass in Java.defaultYearintegerYear is accepted as an optional field for cases like IRIG time format, and will be assumed as current year if not provided.
defaultDayOfYearintegerDefault day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.
ingestTagMetadataIngestTagMetadata
Fields of
IngestTagMetadatatagColumnsmap ofTagNameto stringA map of tag names to column names to derive the tag values from.
additionalFileTagsmap ofTagNameto string
originFilePathsarray of string
ingestJobRidstring (RID)
ingestTransformRidstring (RID)The ingest transform within ingestJobRid that produced this file. An ingest job may fan out to many transforms, so this pins the file to the specific unit of work (and extractor) that created it — resolvable via IngestJobService#getIngestTransform. Extractor logs are tagged with this transform’s UUID (the RID locator), so this also links a file to the logs of the transform that produced it. Absent for directly uploaded files and for files ingested before this link was recorded.
deletedAtstring (date-time)Timestamp that the file is deleted at, only present if the file has been deleted.
metadataDatasetFileMetadataFile-type-specific metadata. For video files, contains timestamp manifest and segment metadata.
Set
typeto"video".videoVideoFileMetadata
Fields of
VideoFileMetadatatimestampManifestVideoFileTimestampManifestSpecifies how to determine absolute timestamps for each frame in the video. Can be an embedded MCAP manifest, an external sidecar file, or calculated from a starting offset applied to presentation timestamps.
Set
typeto"mcap".mcapMcapTimestampManifest
Fields of
McapTimestampManifestmcapChannelLocatorMcapChannelLocator
Set
typeto"topic".topicstring
Set
typeto"id".idinteger
Set
typeto"s3path".s3pathstring
Set
typeto"s3paths".s3pathsarray of stringMultiple sidecar manifest files whose per-frame timestamps are concatenated in list order. Used when a video’s frame timestamps are recorded across several files.
Set
typeto"noManifest".noManifestNoTimestampManifest
Fields of
NoTimestampManifeststartingTimestampTimestampSpecifies the original starting timestamp of the video.
scaleParameterScaleParameterA field that specifies that the frame rate of the video does not match the frame rate of the camera | i.e. a slowed down or sped up video. Can specify either the camera frame rate or the absolute end time.
Set
typeto"trueFrameRate".trueFrameRatenumber
Set
typeto"endingTimestamp".endingTimestampTimestamp
Set
typeto"scaleFactor".scaleFactornumberthe scale factor can be used to calculate whether media duration differs from a video’s | real duration, and if so, the true frame rate of the camera. The video time will thus be scaled | by the ratio of the real duration to media duration, or media frame rate to true frame rate.
segmentMetadataVideoSegmentsMetadataCached aggregate metadata about the segments comprising this video file after segmentation has completed. Includes frame counts, duration, and frame rate. Empty until segmentation is complete. Note: Min/max timestamps are stored in DatasetFile.bounds field. Raw file size is stored in DatasetFile.fileSizeBytes.
Fields of
VideoSegmentsMetadatanumFramesintegerTotal number of frames across all segments.
numSegmentsintegerNumber of segments the video was split into.
scaleFactornumberScale factor applied to timestamps during segmentation.
mediaDurationSecondsnumberTotal duration of the video in seconds.
mediaFrameRatenumberAverage frame rate (FPS) calculated as total frames / duration.
Request
curl -X PUT "https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/file/<file id>/error" \
-H "Authorization: Bearer <api key>" \
-H "Content-Type: application/json" \
--data '{
"errorResult": {
"errorType": "<error type>",
"message": "<message>"
}
}'
import requests
response = requests.put(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/file/<file id>/error",
headers={"Authorization": "Bearer <api key>"},
json={"errorResult": {"errorType": "<error type>", "message": "<message>"}},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.put(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/file/<file id>/error",
headers={"Authorization": "Bearer <api key>"},
json={"errorResult": {"errorType": "<error type>", "message": "<message>"}},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.put(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/file/<file id>/error",
headers={"Authorization": "Bearer <api key>"},
json={"errorResult": {"errorType": "<error type>", "message": "<message>"}},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"id": "<dataset file id>",
"datasetRid": "<dataset rid>",
"name": "<name>",
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"fileSizeBytes": 1,
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"uploadedAt": "2026-01-01T00:00:00Z",
"ingestedAt": "2026-01-01T00:00:00Z",
"ingestStatus": {
"type": "success",
"success": {}
},
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS",
"offset": "2026-01-01T00:00:00Z"
}
}
},
"ingestTagMetadata": {
"tagColumns": {
"<key>": "<column name>"
},
"additionalFileTags": {
"<key>": "<tag value>"
}
},
"originFilePaths": [
"<origin file paths>"
],
"ingestJobRid": "<ingest job rid>",
"ingestTransformRid": "<ingest transform rid>",
"deletedAt": "2026-01-01T00:00:00Z",
"metadata": {
"type": "video",
"video": {
"timestampManifest": {
"type": "mcap",
"mcap": {
"mcapChannelLocator": {
"type": "topic",
"topic": "<mcap channel topic>"
}
}
},
"segmentMetadata": {
"numFrames": 1,
"numSegments": 1,
"scaleFactor": 1,
"mediaDurationSeconds": 1,
"mediaFrameRate": 1
}
}
}
}
Update dataset file metadata#
PUT /catalog/v1/datasets/{datasetRid}/file/{fileId}/metadata
Updates file-type-specific metadata for a dataset file. Currently used to update video segment metadata after video segmentation is complete.
Path parameters
datasetRidstring (RID) required
fileIdstring (UUID) required
Body
A DatasetFileMetadata union.
Set type to "video".
videoVideoFileMetadata
Fields of
VideoFileMetadatatimestampManifestVideoFileTimestampManifestrequiredSpecifies how to determine absolute timestamps for each frame in the video. Can be an embedded MCAP manifest, an external sidecar file, or calculated from a starting offset applied to presentation timestamps.
Set
typeto"mcap".mcapMcapTimestampManifest
Fields of
McapTimestampManifestmcapChannelLocatorMcapChannelLocatorrequired
Set
typeto"topic".topicstring
Set
typeto"id".idinteger
Set
typeto"s3path".s3pathstring
Set
typeto"s3paths".s3pathsarray of stringMultiple sidecar manifest files whose per-frame timestamps are concatenated in list order. Used when a video’s frame timestamps are recorded across several files.
Set
typeto"noManifest".noManifestNoTimestampManifest
Fields of
NoTimestampManifeststartingTimestampTimestamprequiredSpecifies the original starting timestamp of the video.
Fields of
Timestampsecondsinteger (int64) required
nanosinteger (int64) required
scaleParameterScaleParameterA field that specifies that the frame rate of the video does not match the frame rate of the camera | i.e. a slowed down or sped up video. Can specify either the camera frame rate or the absolute end time.
Set
typeto"trueFrameRate".trueFrameRatenumber
Set
typeto"endingTimestamp".endingTimestampTimestamp
Set
typeto"scaleFactor".scaleFactornumberthe scale factor can be used to calculate whether media duration differs from a video’s | real duration, and if so, the true frame rate of the camera. The video time will thus be scaled | by the ratio of the real duration to media duration, or media frame rate to true frame rate.
segmentMetadataVideoSegmentsMetadataCached aggregate metadata about the segments comprising this video file after segmentation has completed. Includes frame counts, duration, and frame rate. Empty until segmentation is complete. Note: Min/max timestamps are stored in DatasetFile.bounds field. Raw file size is stored in DatasetFile.fileSizeBytes.
Fields of
VideoSegmentsMetadatanumFramesinteger requiredTotal number of frames across all segments.
numSegmentsinteger requiredNumber of segments the video was split into.
scaleFactornumber requiredScale factor applied to timestamps during segmentation.
mediaDurationSecondsnumber requiredTotal duration of the video in seconds.
mediaFrameRatenumber requiredAverage frame rate (FPS) calculated as total frames / duration.
Response
idstring (UUID)
datasetRidstring (RID)
namestring
handleHandle
Set
typeto"s3".s3S3Handle
Fields of
S3Handlebucketstring
keystring
fileSizeBytesinteger (int64)Size in bytes of the stored dataset file; for transforming ingests this is the produced output, not the original upload.
boundsBounds
Fields of
BoundsstartTimestamp
Fields of
Timestampsecondsinteger (int64)
nanosinteger (int64)
endTimestamp
typedatasource.TimestampType
Values of
datasource.TimestampTypeRELATIVE
ABSOLUTE
uploadedAtstring (date-time)Timestamp that the file was received and stored, but not processed or made available to consumers.
ingestedAtstring (date-time)Timestamp that the file is ingested at and made available for processing. If the file has failed to be ingested for any reason or is still being processed, then this value will be empty.
ingestStatusIngestStatusV2Ingest status. The in-progress sub-phases (queued, parsing, ingesting) are surfaced as inProgress with the phase carried in InProgressResult.inProgressDetails, so consumers see a stable top-level lifecycle.
Set
typeto"success".successSuccessResult
Set
typeto"error".errorErrorResult
Fields of
ErrorResulterrorTypestring
messagestring
errorInstanceIdstring
Set
typeto"inProgress".inProgressInProgressResult
Fields of
InProgressResultinProgressDetailsInProgressDetails
Set
typeto"deletionInProgress".deletionInProgressDeletionInProgress
Set
typeto"deleted".deletedDeleted
Set
typeto"queued".queuedQueued
Set
typeto"parsing".parsingParsing
Set
typeto"ingesting".ingestingIngesting
timestampMetadataTimestampMetadata
Fields of
TimestampMetadataseriesNamestring
timestampTypecatalog.TimestampType
Set
typeto"relative".relativeRelativeTimestamp
Fields of
RelativeTimestamptimeUnitTimeUnit
Values of
TimeUnitDAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
offsetstring (date-time)
Set
typeto"absolute".absoluteAbsoluteTimestamp
Set
typeto"iso8601".iso8601Iso8601Timestamp
Set
typeto"epochOfTimeUnit".epochOfTimeUnitEpochTimestamp
Fields of
EpochTimestamptimeUnitTimeUnit
Set
typeto"customFormat".customFormatCustomTimestamp
Fields of
CustomTimestampformatstringThe format string should be in the format of the
DateTimeFormatterclass in Java.defaultYearintegerYear is accepted as an optional field for cases like IRIG time format, and will be assumed as current year if not provided.
defaultDayOfYearintegerDefault day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.
ingestTagMetadataIngestTagMetadata
Fields of
IngestTagMetadatatagColumnsmap ofTagNameto stringA map of tag names to column names to derive the tag values from.
additionalFileTagsmap ofTagNameto string
originFilePathsarray of string
ingestJobRidstring (RID)
ingestTransformRidstring (RID)The ingest transform within ingestJobRid that produced this file. An ingest job may fan out to many transforms, so this pins the file to the specific unit of work (and extractor) that created it — resolvable via IngestJobService#getIngestTransform. Extractor logs are tagged with this transform’s UUID (the RID locator), so this also links a file to the logs of the transform that produced it. Absent for directly uploaded files and for files ingested before this link was recorded.
deletedAtstring (date-time)Timestamp that the file is deleted at, only present if the file has been deleted.
metadataDatasetFileMetadataFile-type-specific metadata. For video files, contains timestamp manifest and segment metadata.
Set
typeto"video".videoVideoFileMetadata
Fields of
VideoFileMetadatatimestampManifestVideoFileTimestampManifestSpecifies how to determine absolute timestamps for each frame in the video. Can be an embedded MCAP manifest, an external sidecar file, or calculated from a starting offset applied to presentation timestamps.
Set
typeto"mcap".mcapMcapTimestampManifest
Fields of
McapTimestampManifestmcapChannelLocatorMcapChannelLocator
Set
typeto"topic".topicstring
Set
typeto"id".idinteger
Set
typeto"s3path".s3pathstring
Set
typeto"s3paths".s3pathsarray of stringMultiple sidecar manifest files whose per-frame timestamps are concatenated in list order. Used when a video’s frame timestamps are recorded across several files.
Set
typeto"noManifest".noManifestNoTimestampManifest
Fields of
NoTimestampManifeststartingTimestampTimestampSpecifies the original starting timestamp of the video.
scaleParameterScaleParameterA field that specifies that the frame rate of the video does not match the frame rate of the camera | i.e. a slowed down or sped up video. Can specify either the camera frame rate or the absolute end time.
Set
typeto"trueFrameRate".trueFrameRatenumber
Set
typeto"endingTimestamp".endingTimestampTimestamp
Set
typeto"scaleFactor".scaleFactornumberthe scale factor can be used to calculate whether media duration differs from a video’s | real duration, and if so, the true frame rate of the camera. The video time will thus be scaled | by the ratio of the real duration to media duration, or media frame rate to true frame rate.
segmentMetadataVideoSegmentsMetadataCached aggregate metadata about the segments comprising this video file after segmentation has completed. Includes frame counts, duration, and frame rate. Empty until segmentation is complete. Note: Min/max timestamps are stored in DatasetFile.bounds field. Raw file size is stored in DatasetFile.fileSizeBytes.
Fields of
VideoSegmentsMetadatanumFramesintegerTotal number of frames across all segments.
numSegmentsintegerNumber of segments the video was split into.
scaleFactornumberScale factor applied to timestamps during segmentation.
mediaDurationSecondsnumberTotal duration of the video in seconds.
mediaFrameRatenumberAverage frame rate (FPS) calculated as total frames / duration.
Request
curl -X PUT "https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/file/<file id>/metadata" \
-H "Authorization: Bearer <api key>" \
-H "Content-Type: application/json" \
--data '{
"type": "video",
"video": {
"timestampManifest": {
"type": "mcap",
"mcap": {
"mcapChannelLocator": {
"type": "topic",
"topic": "<mcap channel topic>"
}
}
}
}
}'
import requests
response = requests.put(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/file/<file id>/metadata",
headers={"Authorization": "Bearer <api key>"},
json={
"type": "video",
"video": {
"timestampManifest": {
"type": "mcap",
"mcap": {
"mcapChannelLocator": {
"type": "topic",
"topic": "<mcap channel topic>",
},
},
},
},
},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.put(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/file/<file id>/metadata",
headers={"Authorization": "Bearer <api key>"},
json={
"type": "video",
"video": {
"timestampManifest": {
"type": "mcap",
"mcap": {
"mcapChannelLocator": {
"type": "topic",
"topic": "<mcap channel topic>",
},
},
},
},
},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.put(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/file/<file id>/metadata",
headers={"Authorization": "Bearer <api key>"},
json={
"type": "video",
"video": {
"timestampManifest": {
"type": "mcap",
"mcap": {
"mcapChannelLocator": {
"type": "topic",
"topic": "<mcap channel topic>",
},
},
},
},
},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"id": "<dataset file id>",
"datasetRid": "<dataset rid>",
"name": "<name>",
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"fileSizeBytes": 1,
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"uploadedAt": "2026-01-01T00:00:00Z",
"ingestedAt": "2026-01-01T00:00:00Z",
"ingestStatus": {
"type": "success",
"success": {}
},
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS",
"offset": "2026-01-01T00:00:00Z"
}
}
},
"ingestTagMetadata": {
"tagColumns": {
"<key>": "<column name>"
},
"additionalFileTags": {
"<key>": "<tag value>"
}
},
"originFilePaths": [
"<origin file paths>"
],
"ingestJobRid": "<ingest job rid>",
"ingestTransformRid": "<ingest transform rid>",
"deletedAt": "2026-01-01T00:00:00Z",
"metadata": {
"type": "video",
"video": {
"timestampManifest": {
"type": "mcap",
"mcap": {
"mcapChannelLocator": {
"type": "topic",
"topic": "<mcap channel topic>"
}
}
},
"segmentMetadata": {
"numFrames": 1,
"numSegments": 1,
"scaleFactor": 1,
"mediaDurationSeconds": 1,
"mediaFrameRate": 1
}
}
}
}
Update dataset metadata#
PUT /catalog/v1/datasets/{datasetRid}
Path parameters
datasetRidstring (RID) required
Body
namestring
descriptionstring
labelsarray of string
propertiesmap ofPropertyNameto string deprecatedUser-defined dataset properties. When supplied, the UTF-8 JSON serialization must not exceed 2,000 bytes. Deprecated: use typedProperties
typedPropertiesmap ofPropertyNametoTypedPropertyValue
Set
typeto"stringValue".stringValuestring
Set
typeto"numericValue".numericValuenumber
channelSearchSplitTagKeysarray of stringTag keys whose distinct values channel search surfaces as separate series. Absent leaves the current keys unchanged; an empty set clears them. Every key must be produced by the dataset’s
derivedDefinition.defaultGroupByAndAggregationConfigDefaultGroupByAndAggregationConfigIf set, replaces the dataset’s default group-by and aggregation config in full.
Fields of
DefaultGroupByAndAggregationConfignumericAggregationNumericAggregationOperatorIf set, the default aggregation applied to merge points of a numeric series that share an exact timestamp.
Set
typeto"sum".sumSummation
Set
typeto"average".averageAverage
Set
typeto"min".minMinimum
Set
typeto"max".maxMaximum
Set
typeto"count".countCount
Set
typeto"standardDeviation".standardDeviationStandardDeviation
Set
typeto"rootMeanSquare".rootMeanSquareRootMeanSquare
Set
typeto"percentile".percentilePercentile
Fields of
PercentilepercentileDoubleConstantrequiredPercentile to compute, in range [0, 1]. E.g. 0.5 for median, 0.95 for p95.
Set
typeto"literal".literalnumberCreate from a literal floating-point value.
Set
typeto"variable".variablestringCreate from a named reference name.
Set
typeto"builtin".builtinBuiltinConstantCreate from a named builtin constant; the server resolves the numeric value.
Set
typeto"math".mathMathConstant
Values of
MathConstantPIThe circle constant π (dimensionless).
TAUThe circle constant τ = 2π (dimensionless).
EEuler’s number e (dimensionless).
Set
typeto"physical".physicalPhysicalConstant
Values of
PhysicalConstantSPEED_OF_LIGHTSpeed of light in vacuum, 299792458 m/s (exact).
AVOGADROAvogadro constant, 6.02214076e23 mol^-1 (exact).
BOLTZMANNBoltzmann constant, 1.380649e-23 J/K (exact).
PLANCKPlanck constant, 6.62607015e-34 J*s (exact).
ELEMENTARY_CHARGEElementary charge, 1.602176634e-19 C (exact).
GAS_CONSTANTMolar gas constant, 8.31446261815324 J/(mol*K) (exact, Avogadro times Boltzmann).
STANDARD_GRAVITYStandard acceleration of gravity, 9.80665 m/s^2 (exact by convention).
STANDARD_ATMOSPHEREStandard atmosphere, 101325 Pa (exact by convention).
ABSOLUTE_ZERO_CELSIUSAbsolute zero expressed in degrees Celsius, -273.15 (exact).
GRAVITATIONAL_CONSTANTNewtonian constant of gravitation, 6.67430e-11 m^3/(kg*s^2) (CODATA 2022, measured).
STEFAN_BOLTZMANNStefan-Boltzmann constant, 5.670374419e-8 W/(m^2*K^4) (exact derived).
Set
typeto"udf".udfNumericAggregationUdf
Fields of
NumericAggregationUdfsourceUdfSourcerequired
Set
typeto"rust".rustRustUdfSource
Fields of
RustUdfSourcesourceCodestring requiredRust source code to transpile into WASM. The operation consuming this source defines the entrypoint signature: scalar UDFs receive one scalar per input series, while aggregation UDFs receive grouped value vectors. If several functions appear at the top level, annotate the entrypoint with
#[main].
Response
ridstring (RID)
uuidstring (UUID)
namestring
descriptionstring
displayNamestring
metadatamap of string to string deprecatedDeprecated: Deprecated in favor of properties.
handleHandledeprecatedDeprecated: Deprecated. Use DatasetFile#handle
Set
typeto"s3".s3S3Handle
Fields of
S3Handlebucketstring
keystring
ingestDatestring (date-time)Dataset creation time. The name is historical; use lastIngestedAt for when data was last added.
lastIngestedAtstring (date-time)When data was last added to this dataset, as the latest of a successful file ingest and streaming-session activity. Empty when the dataset has never received either.
ingestStatusIngestStatusdeprecatedDeprecated: Deprecated. Use lastIngestStatus.
Values of
IngestStatusIN_PROGRESS
COMPLETED
FAILED
DELETION_IN_PROGRESS
DELETED
originMetadataDatasetOriginMetadata
Fields of
DatasetOriginMetadatapathstringDeprecated in favor of FileOriginMetadata
xSeriesIsAbsolutebooleanDeprecated in favor of FileOriginMetadata
schemaDirectivePathstringDeprecated in favor of FileOriginMetadata
xSeriesColumnNamestringDeprecated in favor of FileOriginMetadata
xSeriesTimeUnitTimeUnitDeprecated in favor of FileOriginMetadata
Values of
TimeUnitDAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
timestampMetadataTimestampMetadataDeprecated in favor of FileOriginMetadata
Fields of
TimestampMetadataseriesNamestring
timestampTypecatalog.TimestampType
channelConfigChannelConfig
Fields of
ChannelConfigprefixTreeDelimiterstringIf set, will construct a prefix tree for channels of the dataset using the given delimiter.
lastIngestStatusIngestStatusV2
Set
typeto"success".successSuccessResult
Set
typeto"error".errorErrorResult
Fields of
ErrorResulterrorTypestring
messagestring
errorInstanceIdstring
Set
typeto"inProgress".inProgressInProgressResult
Fields of
InProgressResultinProgressDetailsInProgressDetails
Set
typeto"deletionInProgress".deletionInProgressDeletionInProgress
Set
typeto"deleted".deletedDeleted
Set
typeto"queued".queuedQueued
Set
typeto"parsing".parsingParsing
Set
typeto"ingesting".ingestingIngesting
Fields of
IngestingprogressnumberOptional fraction between 0 and 1 indicating progress through ingestion.
retentionPolicyRetentionPolicy
Fields of
RetentionPolicytypeRetentionPolicyType
Values of
RetentionPolicyTypeKEEP_UNTIL_EXPIRY
KEEP_FOREVER
expiryDaysinteger
sourcestring
boundsBounds
Fields of
BoundsstartTimestamp
Fields of
Timestampsecondsinteger (int64)
nanosinteger (int64)
endTimestamp
typedatasource.TimestampType
Values of
datasource.TimestampTypeRELATIVE
ABSOLUTE
timestampTypeWeakTimestampType
Values of
WeakTimestampTypeABSOLUTE
RELATIVE
PENDING
labelsarray of string
propertiesmap ofPropertyNameto string deprecatedDeprecated: use typedProperties
typedPropertiesmap ofPropertyNametoTypedPropertyValue
Set
typeto"stringValue".stringValuestring
Set
typeto"numericValue".numericValuenumber
granularityGranularity
Values of
GranularityNANOSECONDS
allowStreamingboolean
isArchivedboolean
defaultGroupByAndAggregationConfigDefaultGroupByAndAggregationConfig
Fields of
DefaultGroupByAndAggregationConfignumericAggregationNumericAggregationOperatorIf set, the default aggregation applied to merge points of a numeric series that share an exact timestamp.
Set
typeto"sum".sumSummation
Set
typeto"average".averageAverage
Set
typeto"min".minMinimum
Set
typeto"max".maxMaximum
Set
typeto"count".countCount
Set
typeto"standardDeviation".standardDeviationStandardDeviation
Set
typeto"rootMeanSquare".rootMeanSquareRootMeanSquare
Set
typeto"percentile".percentilePercentile
Set
typeto"udf".
migratedFromConnectionRidstring (RID)Connection RID whose data source was migrated into this dataset. Empty for datasets that were not created by the legacy connection-to-dataset migrations, or when the original connection row no longer exists.
externalConnectionMetadataExternalConnectionMetadata
Fields of
ExternalConnectionMetadataconnectionDetailsConnectionDetails
Set
typeto"timescale".timescaleTimescaleConnectionDetails
Set
typeto"influx".influxInflux2ConnectionDetails
Set
typeto"influx1".influx1Influx1ConnectionDetails
Set
typeto"nominal".nominalNominalConnectionDetails
Set
typeto"timestream".timestreamTimestreamConnectionDetails
Set
typeto"visualCrossing".visualCrossingVisualCrossingConnectionDetails
Set
typeto"bigQuery".bigQueryBigQueryConnectionDetails
connectionStatusConnectionStatus
Fields of
ConnectionStatusisConnectedboolean
limitsLimitsConfig
Fields of
LimitsConfigmaxQueryRangeconnection.api.Duration
derivedDefinitionDerivedDefinitionDefinition for a derived dataset. When present, this dataset is virtual and backed by the referenced provided Dataset definition instead of persisted dataset files.
Fields of
DerivedDefinitionspecapi.Dataset
Set
typeto"asset".assetAssetSingle asset. Expands to one grouping per data scope on the asset. Groupings are tagged by
assetRidanddataScopeso callers can safely compose multiple assets withcombine.
Set
typeto"run".runRunSingle run. Expands to one grouping per data source attached to the run: asset-typed sources contribute one grouping per data scope on the asset; dataset-typed sources contribute one grouping per dataset. Groupings are tagged by
runRidanddataScope; asset-typed groupings additionally carryassetRidso callers can safely compose multiple runs withcombine.
Set
typeto"saved".savedSavedDataset
Set
typeto"search".searchSearchDataset
Set
typeto"combine".combineCombinedDataset
Set
typeto"tag".tagTaggedDataset
Set
typeto"filter".filterFilteredDataset
Set
typeto"timeShift".timeShiftTimeShiftedDataset
Set
typeto"alignByAnchor".alignByAnchorAnchorAlignedDataset
Set
typeto"withSeries".withSeriesWithSeriesDataset
Set
typeto"dropSeries".dropSeriesDropSeriesDataset
Set
typeto"reference".referenceDatasetReference
Set
typeto"withDimensions".withDimensionsDimensionsDataset
commitCommit
Fields of
Commitidstring
resourceRidstring (RID)
parentCommitstring
mergeParentCommitstring
messagestring
isWorkingStatebooleanA working state commit is created via the
saveWorkingStateendpoint and is non-permanent. In the future, it may be compacted and not exist.committedBystring (RID)
committedAtstring (date-time)
channelSearchSplitTagKeysarray of stringTag keys whose distinct values channel search surfaces as separate series. Each listed key is retained on this dataset’s channel-search locators, so a channel carrying two values of that key appears once per value instead of collapsing into a single entry — the behavior a data scope has on an asset. Keys not listed here stay available for filtering and grouping but do not contribute to series cardinality. Every key must be produced by this dataset’s
derivedDefinition. Ingested tag keys cannot be split on.
Request
curl -X PUT "https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>" \
-H "Authorization: Bearer <api key>" \
-H "Content-Type: application/json" \
--data '{
"name": "<name>",
"description": "<description>",
"labels": [
"<label>"
],
"properties": {
"<key>": "<property value>"
},
"typedProperties": {
"<key>": {
"type": "stringValue",
"stringValue": "<property value>"
}
},
"channelSearchSplitTagKeys": [
"<tag name>"
],
"defaultGroupByAndAggregationConfig": {
"numericAggregation": {
"type": "sum",
"sum": {}
}
}
}'
import requests
response = requests.put(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>",
headers={"Authorization": "Bearer <api key>"},
json={
"name": "<name>",
"description": "<description>",
"labels": ["<label>"],
"properties": {"<key>": "<property value>"},
"typedProperties": {
"<key>": {"type": "stringValue", "stringValue": "<property value>"},
},
"channelSearchSplitTagKeys": ["<tag name>"],
"defaultGroupByAndAggregationConfig": {
"numericAggregation": {"type": "sum", "sum": {}},
},
},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.put(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>",
headers={"Authorization": "Bearer <api key>"},
json={
"name": "<name>",
"description": "<description>",
"labels": ["<label>"],
"properties": {"<key>": "<property value>"},
"typedProperties": {
"<key>": {"type": "stringValue", "stringValue": "<property value>"},
},
"channelSearchSplitTagKeys": ["<tag name>"],
"defaultGroupByAndAggregationConfig": {
"numericAggregation": {"type": "sum", "sum": {}},
},
},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.put(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>",
headers={"Authorization": "Bearer <api key>"},
json={
"name": "<name>",
"description": "<description>",
"labels": ["<label>"],
"properties": {"<key>": "<property value>"},
"typedProperties": {
"<key>": {"type": "stringValue", "stringValue": "<property value>"},
},
"channelSearchSplitTagKeys": ["<tag name>"],
"defaultGroupByAndAggregationConfig": {
"numericAggregation": {"type": "sum", "sum": {}},
},
},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"rid": "<rid>",
"uuid": "<uuid>",
"name": "<name>",
"description": "<description>",
"displayName": "<display name>",
"metadata": {
"<key>": "<value>"
},
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"ingestDate": "2026-01-01T00:00:00Z",
"lastIngestedAt": "2026-01-01T00:00:00Z",
"ingestStatus": "IN_PROGRESS",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": true,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS",
"offset": "2026-01-01T00:00:00Z"
}
}
},
"channelConfig": {
"prefixTreeDelimiter": "<prefix tree delimiter>"
}
},
"lastIngestStatus": {
"type": "success",
"success": {}
},
"retentionPolicy": {
"type": "KEEP_UNTIL_EXPIRY",
"expiryDays": 1
},
"source": "<source>",
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"timestampType": "ABSOLUTE",
"labels": [
"<label>"
],
"properties": {
"<key>": "<property value>"
},
"typedProperties": {
"<key>": {
"type": "stringValue",
"stringValue": "<property value>"
}
},
"granularity": "NANOSECONDS",
"allowStreaming": true,
"isArchived": true,
"defaultGroupByAndAggregationConfig": {
"numericAggregation": {
"type": "sum",
"sum": {}
}
},
"migratedFromConnectionRid": "<connection rid>",
"externalConnectionMetadata": {
"connectionDetails": {
"type": "timescale",
"timescale": {
"database": "<database>",
"host": "<host>",
"port": 1,
"username": "<username>",
"passwordSecretRid": "<secret rid>"
}
},
"connectionStatus": {
"isConnected": true
},
"limits": {
"maxQueryRange": {
"unit": "HOURS",
"count": 1
}
}
},
"derivedDefinition": {
"spec": {
"type": "asset",
"asset": {
"rid": {
"type": "literal",
"literal": "<literal>"
}
}
},
"commit": {
"id": "<commit id>",
"resourceRid": "<resource rid>",
"parentCommit": "<commit id>",
"mergeParentCommit": "<commit id>",
"message": "<message>",
"isWorkingState": true,
"committedBy": "<user rid>",
"committedAt": "2026-01-01T00:00:00Z"
}
},
"channelSearchSplitTagKeys": [
"<tag name>"
]
}
Get dataset derived definition#
GET /catalog/v1/datasets/{datasetRid}/derived-definition
Returns a versioned derived definition. If commit is omitted, returns the latest commit on main. Where the deployment resolves derived datasets through their same-UUID pipelines, the definition and its commits are the pipeline’s; a commit from the dataset’s own earlier history still resolves.
Path parameters
datasetRidstring (RID) required
Query parameters
commitstring
Response
specapi.Dataset
Set
typeto"asset".assetAssetSingle asset. Expands to one grouping per data scope on the asset. Groupings are tagged by
assetRidanddataScopeso callers can safely compose multiple assets withcombine.Fields of
AssetridStringConstantResource identifier of an asset
Set
typeto"run".runRunSingle run. Expands to one grouping per data source attached to the run: asset-typed sources contribute one grouping per data scope on the asset; dataset-typed sources contribute one grouping per dataset. Groupings are tagged by
runRidanddataScope; asset-typed groupings additionally carryassetRidso callers can safely compose multiple runs withcombine.Fields of
RunridStringConstantResource identifier of a run
Set
typeto"saved".savedSavedDataset
Fields of
SavedDatasetridStringConstantA data source RID, given as a literal or as a variable resolved from
Context. A pipeline’s logic only accepts catalog datasets/pipelines, since it is persisted without context.
Set
typeto"search".searchSearchDataset
Fields of
SearchDatasettargetSearchTargetWhether to search assets or runs and with which query.
maxResultsintegerMaximum number of assets or runs returned from search. Defaults to 10000 when omitted.
Set
typeto"combine".combineCombinedDataset
Fields of
CombinedDatasetinputsarray ofapi.DatasetThe datasets to combine.
Set
typeto"tag".tagTaggedDataset
Fields of
TaggedDatasetinputapi.DatasetThe underlying dataset to tag.
keyStringConstantTag key to inject onto each series in the dataset.
valueStringConstantTag value to assign to the injected key.
Set
typeto"filter".filterFilteredDataset
Fields of
FilteredDatasetinputapi.DatasetThe underlying dataset to filter.
predicateTagPredicateConjunctive tag predicate used to keep matching series.
Set
typeto"timeShift".timeShiftTimeShiftedDataset
Fields of
TimeShiftedDatasetinputapi.DatasetThe underlying dataset to time-shift.
offsetcompute.api.DurationThe time shift to apply. Positive values shift data forward in time, negative values shift backward.
Set
typeto"alignByAnchor".alignByAnchorAnchorAlignedDataset
Fields of
AnchorAlignedDatasetinputapi.DatasetThe dataset to align.
anchorDatasetAnchorSelects the timestamp each anchored portion of the input is aligned by.
referenceAnchorReferenceWhen present, each anchor is aligned onto the referenced anchor timestamp instead of onto timestamp zero, so output remains in the input’s absolute time domain and the referenced portion is unshifted. When absent, the selected anchor becomes timestamp zero.
Set
typeto"withSeries".withSeriesWithSeriesDataset
Fields of
WithSeriesDatasetinputapi.DatasetThe base dataset to which the new series will be added.
seriesarray ofNamedSeriesThe named derived series to add.
Set
typeto"dropSeries".dropSeriesDropSeriesDataset
Fields of
DropSeriesDatasetinputapi.DatasetThe base dataset to drop series from.
namesarray ofStringConstantNames of the series to drop.
Set
typeto"reference".referenceDatasetReference
Fields of
DatasetReferencenameStringConstantName of the dataset reference to resolve from the context.
Set
typeto"withDimensions".withDimensionsDimensionsDataset
Fields of
DimensionsDatasetinputapi.DatasetThe underlying dataset to tag.
dimensionsarray ofDimensionThe dimensions to inject. Must not be empty, must have at most 10 entries, and every dimension must have a distinct tag key. A tag key must not be a
ReservedTagKey, start withEVENT_LABEL_PREFIX:, or start withNOMINAL_.
commitCommit
Fields of
Commitidstring
resourceRidstring (RID)
parentCommitstring
mergeParentCommitstring
messagestring
isWorkingStatebooleanA working state commit is created via the
saveWorkingStateendpoint and is non-permanent. In the future, it may be compacted and not exist.committedBystring (RID)
committedAtstring (date-time)
Request
curl "https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/derived-definition" \
-H "Authorization: Bearer <api key>"
import requests
response = requests.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/derived-definition",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/derived-definition",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/derived-definition",
headers={"Authorization": "Bearer <api key>"},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"spec": {
"type": "asset",
"asset": {
"rid": {
"type": "literal",
"literal": "<literal>"
}
}
},
"commit": {
"id": "<commit id>",
"resourceRid": "<resource rid>",
"parentCommit": "<commit id>",
"mergeParentCommit": "<commit id>",
"message": "<message>",
"isWorkingState": true,
"committedBy": "<user rid>",
"committedAt": "2026-01-01T00:00:00Z"
}
}
Create dataset derived definition#
POST /catalog/v1/datasets/{datasetRid}/derived-definition
Initializes a derived definition on a dataset that is not yet derived and has no ingest or streaming state (files, create-time handle, bounds, external connection, or streaming sessions). Subsequent changes use commitDerivedDefinition. Throws if the dataset is already derived, is non-empty, referenced datasets are unauthorized, or the definition is nested/unsupported. Where the deployment resolves derived datasets through their pipelines, the same-UUID pipeline is created alongside, and later commits go to it.
Path parameters
datasetRidstring (RID) required
Body
specapi.DatasetrequiredInitial derived dataset logic.
Set
typeto"asset".assetAssetSingle asset. Expands to one grouping per data scope on the asset. Groupings are tagged by
assetRidanddataScopeso callers can safely compose multiple assets withcombine.Fields of
AssetridStringConstantrequiredResource identifier of an asset
Set
typeto"run".runRunSingle run. Expands to one grouping per data source attached to the run: asset-typed sources contribute one grouping per data scope on the asset; dataset-typed sources contribute one grouping per dataset. Groupings are tagged by
runRidanddataScope; asset-typed groupings additionally carryassetRidso callers can safely compose multiple runs withcombine.Fields of
RunridStringConstantrequiredResource identifier of a run
Set
typeto"saved".savedSavedDataset
Fields of
SavedDatasetridStringConstantrequiredA data source RID, given as a literal or as a variable resolved from
Context. A pipeline’s logic only accepts catalog datasets/pipelines, since it is persisted without context.
Set
typeto"search".searchSearchDataset
Fields of
SearchDatasettargetSearchTargetrequiredWhether to search assets or runs and with which query.
maxResultsintegerMaximum number of assets or runs returned from search. Defaults to 10000 when omitted.
Set
typeto"combine".combineCombinedDataset
Fields of
CombinedDatasetinputsarray ofapi.DatasetThe datasets to combine.
Set
typeto"tag".tagTaggedDataset
Fields of
TaggedDatasetinputapi.DatasetrequiredThe underlying dataset to tag.
keyStringConstantrequiredTag key to inject onto each series in the dataset.
valueStringConstantrequiredTag value to assign to the injected key.
Set
typeto"filter".filterFilteredDataset
Fields of
FilteredDatasetinputapi.DatasetrequiredThe underlying dataset to filter.
predicateTagPredicaterequiredConjunctive tag predicate used to keep matching series.
Set
typeto"timeShift".timeShiftTimeShiftedDataset
Fields of
TimeShiftedDatasetinputapi.DatasetrequiredThe underlying dataset to time-shift.
offsetcompute.api.DurationrequiredThe time shift to apply. Positive values shift data forward in time, negative values shift backward.
Set
typeto"alignByAnchor".alignByAnchorAnchorAlignedDataset
Fields of
AnchorAlignedDatasetinputapi.DatasetrequiredThe dataset to align.
anchorDatasetAnchorrequiredSelects the timestamp each anchored portion of the input is aligned by.
referenceAnchorReferenceWhen present, each anchor is aligned onto the referenced anchor timestamp instead of onto timestamp zero, so output remains in the input’s absolute time domain and the referenced portion is unshifted. When absent, the selected anchor becomes timestamp zero.
Set
typeto"withSeries".withSeriesWithSeriesDataset
Fields of
WithSeriesDatasetinputapi.DatasetrequiredThe base dataset to which the new series will be added.
seriesarray ofNamedSeriesThe named derived series to add.
Set
typeto"dropSeries".dropSeriesDropSeriesDataset
Fields of
DropSeriesDatasetinputapi.DatasetrequiredThe base dataset to drop series from.
namesarray ofStringConstantNames of the series to drop.
Set
typeto"reference".referenceDatasetReference
Fields of
DatasetReferencenameStringConstantrequiredName of the dataset reference to resolve from the context.
Set
typeto"withDimensions".withDimensionsDimensionsDataset
Fields of
DimensionsDatasetinputapi.DatasetrequiredThe underlying dataset to tag.
dimensionsarray ofDimensionThe dimensions to inject. Must not be empty, must have at most 10 entries, and every dimension must have a distinct tag key. A tag key must not be a
ReservedTagKey, start withEVENT_LABEL_PREFIX:, or start withNOMINAL_.
messagestring requiredInitial commit message for the derived definition’s main branch.
Response
specapi.Dataset
Set
typeto"asset".assetAssetSingle asset. Expands to one grouping per data scope on the asset. Groupings are tagged by
assetRidanddataScopeso callers can safely compose multiple assets withcombine.Fields of
AssetridStringConstantResource identifier of an asset
Set
typeto"run".runRunSingle run. Expands to one grouping per data source attached to the run: asset-typed sources contribute one grouping per data scope on the asset; dataset-typed sources contribute one grouping per dataset. Groupings are tagged by
runRidanddataScope; asset-typed groupings additionally carryassetRidso callers can safely compose multiple runs withcombine.Fields of
RunridStringConstantResource identifier of a run
Set
typeto"saved".savedSavedDataset
Fields of
SavedDatasetridStringConstantA data source RID, given as a literal or as a variable resolved from
Context. A pipeline’s logic only accepts catalog datasets/pipelines, since it is persisted without context.
Set
typeto"search".searchSearchDataset
Fields of
SearchDatasettargetSearchTargetWhether to search assets or runs and with which query.
maxResultsintegerMaximum number of assets or runs returned from search. Defaults to 10000 when omitted.
Set
typeto"combine".combineCombinedDataset
Fields of
CombinedDatasetinputsarray ofapi.DatasetThe datasets to combine.
Set
typeto"tag".tagTaggedDataset
Fields of
TaggedDatasetinputapi.DatasetThe underlying dataset to tag.
keyStringConstantTag key to inject onto each series in the dataset.
valueStringConstantTag value to assign to the injected key.
Set
typeto"filter".filterFilteredDataset
Fields of
FilteredDatasetinputapi.DatasetThe underlying dataset to filter.
predicateTagPredicateConjunctive tag predicate used to keep matching series.
Set
typeto"timeShift".timeShiftTimeShiftedDataset
Fields of
TimeShiftedDatasetinputapi.DatasetThe underlying dataset to time-shift.
offsetcompute.api.DurationThe time shift to apply. Positive values shift data forward in time, negative values shift backward.
Set
typeto"alignByAnchor".alignByAnchorAnchorAlignedDataset
Fields of
AnchorAlignedDatasetinputapi.DatasetThe dataset to align.
anchorDatasetAnchorSelects the timestamp each anchored portion of the input is aligned by.
referenceAnchorReferenceWhen present, each anchor is aligned onto the referenced anchor timestamp instead of onto timestamp zero, so output remains in the input’s absolute time domain and the referenced portion is unshifted. When absent, the selected anchor becomes timestamp zero.
Set
typeto"withSeries".withSeriesWithSeriesDataset
Fields of
WithSeriesDatasetinputapi.DatasetThe base dataset to which the new series will be added.
seriesarray ofNamedSeriesThe named derived series to add.
Set
typeto"dropSeries".dropSeriesDropSeriesDataset
Fields of
DropSeriesDatasetinputapi.DatasetThe base dataset to drop series from.
namesarray ofStringConstantNames of the series to drop.
Set
typeto"reference".referenceDatasetReference
Fields of
DatasetReferencenameStringConstantName of the dataset reference to resolve from the context.
Set
typeto"withDimensions".withDimensionsDimensionsDataset
Fields of
DimensionsDatasetinputapi.DatasetThe underlying dataset to tag.
dimensionsarray ofDimensionThe dimensions to inject. Must not be empty, must have at most 10 entries, and every dimension must have a distinct tag key. A tag key must not be a
ReservedTagKey, start withEVENT_LABEL_PREFIX:, or start withNOMINAL_.
commitCommit
Fields of
Commitidstring
resourceRidstring (RID)
parentCommitstring
mergeParentCommitstring
messagestring
isWorkingStatebooleanA working state commit is created via the
saveWorkingStateendpoint and is non-permanent. In the future, it may be compacted and not exist.committedBystring (RID)
committedAtstring (date-time)
Request
curl -X POST "https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/derived-definition" \
-H "Authorization: Bearer <api key>" \
-H "Content-Type: application/json" \
--data '{
"spec": {
"type": "asset",
"asset": {
"rid": {
"type": "literal",
"literal": "<literal>"
}
}
},
"message": "<message>"
}'
import requests
response = requests.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/derived-definition",
headers={"Authorization": "Bearer <api key>"},
json={
"spec": {
"type": "asset",
"asset": {"rid": {"type": "literal", "literal": "<literal>"}},
},
"message": "<message>",
},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/derived-definition",
headers={"Authorization": "Bearer <api key>"},
json={
"spec": {
"type": "asset",
"asset": {"rid": {"type": "literal", "literal": "<literal>"}},
},
"message": "<message>",
},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/derived-definition",
headers={"Authorization": "Bearer <api key>"},
json={
"spec": {
"type": "asset",
"asset": {"rid": {"type": "literal", "literal": "<literal>"}},
},
"message": "<message>",
},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"spec": {
"type": "asset",
"asset": {
"rid": {
"type": "literal",
"literal": "<literal>"
}
}
},
"commit": {
"id": "<commit id>",
"resourceRid": "<resource rid>",
"parentCommit": "<commit id>",
"mergeParentCommit": "<commit id>",
"message": "<message>",
"isWorkingState": true,
"committedBy": "<user rid>",
"committedAt": "2026-01-01T00:00:00Z"
}
}
Commit derived definition#
POST /catalog/v1/datasets/{datasetRid}/derived-definition/commit
Replaces the current derived definition by creating a permanent commit on main. Throws if the dataset is not derived, if referenced datasets are unauthorized, or if latestCommit is stale. Where the deployment resolves derived datasets through their same-UUID pipelines, this commits to the pipeline, and latestCommit is checked against the pipeline’s head.
Path parameters
datasetRidstring (RID) required
Body
specapi.Datasetrequired
Set
typeto"asset".assetAssetSingle asset. Expands to one grouping per data scope on the asset. Groupings are tagged by
assetRidanddataScopeso callers can safely compose multiple assets withcombine.Fields of
AssetridStringConstantrequiredResource identifier of an asset
Set
typeto"run".runRunSingle run. Expands to one grouping per data source attached to the run: asset-typed sources contribute one grouping per data scope on the asset; dataset-typed sources contribute one grouping per dataset. Groupings are tagged by
runRidanddataScope; asset-typed groupings additionally carryassetRidso callers can safely compose multiple runs withcombine.Fields of
RunridStringConstantrequiredResource identifier of a run
Set
typeto"saved".savedSavedDataset
Fields of
SavedDatasetridStringConstantrequiredA data source RID, given as a literal or as a variable resolved from
Context. A pipeline’s logic only accepts catalog datasets/pipelines, since it is persisted without context.
Set
typeto"search".searchSearchDataset
Fields of
SearchDatasettargetSearchTargetrequiredWhether to search assets or runs and with which query.
maxResultsintegerMaximum number of assets or runs returned from search. Defaults to 10000 when omitted.
Set
typeto"combine".combineCombinedDataset
Fields of
CombinedDatasetinputsarray ofapi.DatasetThe datasets to combine.
Set
typeto"tag".tagTaggedDataset
Fields of
TaggedDatasetinputapi.DatasetrequiredThe underlying dataset to tag.
keyStringConstantrequiredTag key to inject onto each series in the dataset.
valueStringConstantrequiredTag value to assign to the injected key.
Set
typeto"filter".filterFilteredDataset
Fields of
FilteredDatasetinputapi.DatasetrequiredThe underlying dataset to filter.
predicateTagPredicaterequiredConjunctive tag predicate used to keep matching series.
Set
typeto"timeShift".timeShiftTimeShiftedDataset
Fields of
TimeShiftedDatasetinputapi.DatasetrequiredThe underlying dataset to time-shift.
offsetcompute.api.DurationrequiredThe time shift to apply. Positive values shift data forward in time, negative values shift backward.
Set
typeto"alignByAnchor".alignByAnchorAnchorAlignedDataset
Fields of
AnchorAlignedDatasetinputapi.DatasetrequiredThe dataset to align.
anchorDatasetAnchorrequiredSelects the timestamp each anchored portion of the input is aligned by.
referenceAnchorReferenceWhen present, each anchor is aligned onto the referenced anchor timestamp instead of onto timestamp zero, so output remains in the input’s absolute time domain and the referenced portion is unshifted. When absent, the selected anchor becomes timestamp zero.
Set
typeto"withSeries".withSeriesWithSeriesDataset
Fields of
WithSeriesDatasetinputapi.DatasetrequiredThe base dataset to which the new series will be added.
seriesarray ofNamedSeriesThe named derived series to add.
Set
typeto"dropSeries".dropSeriesDropSeriesDataset
Fields of
DropSeriesDatasetinputapi.DatasetrequiredThe base dataset to drop series from.
namesarray ofStringConstantNames of the series to drop.
Set
typeto"reference".referenceDatasetReference
Fields of
DatasetReferencenameStringConstantrequiredName of the dataset reference to resolve from the context.
Set
typeto"withDimensions".withDimensionsDimensionsDataset
Fields of
DimensionsDatasetinputapi.DatasetrequiredThe underlying dataset to tag.
dimensionsarray ofDimensionThe dimensions to inject. Must not be empty, must have at most 10 entries, and every dimension must have a distinct tag key. A tag key must not be a
ReservedTagKey, start withEVENT_LABEL_PREFIX:, or start withNOMINAL_.
messagestring required
latestCommitstringIf present, validates that the latest commit on main matches this id, and otherwise throws CommitConflict.
Response
specapi.Dataset
Set
typeto"asset".assetAssetSingle asset. Expands to one grouping per data scope on the asset. Groupings are tagged by
assetRidanddataScopeso callers can safely compose multiple assets withcombine.Fields of
AssetridStringConstantResource identifier of an asset
Set
typeto"run".runRunSingle run. Expands to one grouping per data source attached to the run: asset-typed sources contribute one grouping per data scope on the asset; dataset-typed sources contribute one grouping per dataset. Groupings are tagged by
runRidanddataScope; asset-typed groupings additionally carryassetRidso callers can safely compose multiple runs withcombine.Fields of
RunridStringConstantResource identifier of a run
Set
typeto"saved".savedSavedDataset
Fields of
SavedDatasetridStringConstantA data source RID, given as a literal or as a variable resolved from
Context. A pipeline’s logic only accepts catalog datasets/pipelines, since it is persisted without context.
Set
typeto"search".searchSearchDataset
Fields of
SearchDatasettargetSearchTargetWhether to search assets or runs and with which query.
maxResultsintegerMaximum number of assets or runs returned from search. Defaults to 10000 when omitted.
Set
typeto"combine".combineCombinedDataset
Fields of
CombinedDatasetinputsarray ofapi.DatasetThe datasets to combine.
Set
typeto"tag".tagTaggedDataset
Fields of
TaggedDatasetinputapi.DatasetThe underlying dataset to tag.
keyStringConstantTag key to inject onto each series in the dataset.
valueStringConstantTag value to assign to the injected key.
Set
typeto"filter".filterFilteredDataset
Fields of
FilteredDatasetinputapi.DatasetThe underlying dataset to filter.
predicateTagPredicateConjunctive tag predicate used to keep matching series.
Set
typeto"timeShift".timeShiftTimeShiftedDataset
Fields of
TimeShiftedDatasetinputapi.DatasetThe underlying dataset to time-shift.
offsetcompute.api.DurationThe time shift to apply. Positive values shift data forward in time, negative values shift backward.
Set
typeto"alignByAnchor".alignByAnchorAnchorAlignedDataset
Fields of
AnchorAlignedDatasetinputapi.DatasetThe dataset to align.
anchorDatasetAnchorSelects the timestamp each anchored portion of the input is aligned by.
referenceAnchorReferenceWhen present, each anchor is aligned onto the referenced anchor timestamp instead of onto timestamp zero, so output remains in the input’s absolute time domain and the referenced portion is unshifted. When absent, the selected anchor becomes timestamp zero.
Set
typeto"withSeries".withSeriesWithSeriesDataset
Fields of
WithSeriesDatasetinputapi.DatasetThe base dataset to which the new series will be added.
seriesarray ofNamedSeriesThe named derived series to add.
Set
typeto"dropSeries".dropSeriesDropSeriesDataset
Fields of
DropSeriesDatasetinputapi.DatasetThe base dataset to drop series from.
namesarray ofStringConstantNames of the series to drop.
Set
typeto"reference".referenceDatasetReference
Fields of
DatasetReferencenameStringConstantName of the dataset reference to resolve from the context.
Set
typeto"withDimensions".withDimensionsDimensionsDataset
Fields of
DimensionsDatasetinputapi.DatasetThe underlying dataset to tag.
dimensionsarray ofDimensionThe dimensions to inject. Must not be empty, must have at most 10 entries, and every dimension must have a distinct tag key. A tag key must not be a
ReservedTagKey, start withEVENT_LABEL_PREFIX:, or start withNOMINAL_.
commitCommit
Fields of
Commitidstring
resourceRidstring (RID)
parentCommitstring
mergeParentCommitstring
messagestring
isWorkingStatebooleanA working state commit is created via the
saveWorkingStateendpoint and is non-permanent. In the future, it may be compacted and not exist.committedBystring (RID)
committedAtstring (date-time)
Request
curl -X POST "https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/derived-definition/commit" \
-H "Authorization: Bearer <api key>" \
-H "Content-Type: application/json" \
--data '{
"spec": {
"type": "asset",
"asset": {
"rid": {
"type": "literal",
"literal": "<literal>"
}
}
},
"message": "<message>"
}'
import requests
response = requests.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/derived-definition/commit",
headers={"Authorization": "Bearer <api key>"},
json={
"spec": {
"type": "asset",
"asset": {"rid": {"type": "literal", "literal": "<literal>"}},
},
"message": "<message>",
},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/derived-definition/commit",
headers={"Authorization": "Bearer <api key>"},
json={
"spec": {
"type": "asset",
"asset": {"rid": {"type": "literal", "literal": "<literal>"}},
},
"message": "<message>",
},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/derived-definition/commit",
headers={"Authorization": "Bearer <api key>"},
json={
"spec": {
"type": "asset",
"asset": {"rid": {"type": "literal", "literal": "<literal>"}},
},
"message": "<message>",
},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"spec": {
"type": "asset",
"asset": {
"rid": {
"type": "literal",
"literal": "<literal>"
}
}
},
"commit": {
"id": "<commit id>",
"resourceRid": "<resource rid>",
"parentCommit": "<commit id>",
"mergeParentCommit": "<commit id>",
"message": "<message>",
"isWorkingState": true,
"committedBy": "<user rid>",
"committedAt": "2026-01-01T00:00:00Z"
}
}
Update global dataset bounds#
PUT /catalog/v1/datasets/{rid}/bounds-from-streaming
Update the bounds for a dataset without updating bounds of files within the dataset. If the current bounds of the dataset are not set, then the bounds of the request will be used. Otherwise, the bounds will be min(current start, request start), max(current end, request end).
Path parameters
ridstring (RID) required
Body
boundsBoundsrequired
Fields of
BoundsstartTimestamprequired
Fields of
Timestampsecondsinteger (int64) required
nanosinteger (int64) required
endTimestamprequired
typedatasource.TimestampTyperequired
Values of
datasource.TimestampTypeRELATIVE
ABSOLUTE
Response
ridstring (RID)
namestring
handleHandledeprecatedDeprecated: Deprecated. Use DatasetFile#handle
Set
typeto"s3".s3S3Handle
Fields of
S3Handlebucketstring
keystring
descriptionstring
originMetadataDatasetOriginMetadata
Fields of
DatasetOriginMetadatapathstringDeprecated in favor of FileOriginMetadata
xSeriesIsAbsolutebooleanDeprecated in favor of FileOriginMetadata
schemaDirectivePathstringDeprecated in favor of FileOriginMetadata
xSeriesColumnNamestringDeprecated in favor of FileOriginMetadata
xSeriesTimeUnitTimeUnitDeprecated in favor of FileOriginMetadata
Values of
TimeUnitDAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
timestampMetadataTimestampMetadataDeprecated in favor of FileOriginMetadata
Fields of
TimestampMetadataseriesNamestring
timestampTypecatalog.TimestampType
channelConfigChannelConfig
Fields of
ChannelConfigprefixTreeDelimiterstringIf set, will construct a prefix tree for channels of the dataset using the given delimiter.
boundsBounds
Fields of
BoundsstartTimestamp
Fields of
Timestampsecondsinteger (int64)
nanosinteger (int64)
endTimestamp
typedatasource.TimestampType
Values of
datasource.TimestampTypeRELATIVE
ABSOLUTE
propertiesmap ofPropertyNameto string deprecatedDeprecated: use typedProperties
typedPropertiesmap ofPropertyNametoTypedPropertyValue
Set
typeto"stringValue".stringValuestring
Set
typeto"numericValue".numericValuenumber
labelsarray of string
timestampTypeWeakTimestampType
Values of
WeakTimestampTypeABSOLUTE
RELATIVE
PENDING
allowStreamingboolean
granularityGranularity
Values of
GranularityNANOSECONDS
isArchivedboolean
defaultGroupByAndAggregationConfigDefaultGroupByAndAggregationConfig
Fields of
DefaultGroupByAndAggregationConfignumericAggregationNumericAggregationOperatorIf set, the default aggregation applied to merge points of a numeric series that share an exact timestamp.
Set
typeto"sum".sumSummation
Set
typeto"average".averageAverage
Set
typeto"min".minMinimum
Set
typeto"max".maxMaximum
Set
typeto"count".countCount
Set
typeto"standardDeviation".standardDeviationStandardDeviation
Set
typeto"rootMeanSquare".rootMeanSquareRootMeanSquare
Set
typeto"percentile".percentilePercentile
Set
typeto"udf".
derivedDefinitionDerivedDefinitionDefinition for a derived dataset. When present, this dataset is virtual and backed by the referenced provided Dataset definition instead of persisted dataset files.
Fields of
DerivedDefinitionspecapi.Dataset
Set
typeto"asset".assetAssetSingle asset. Expands to one grouping per data scope on the asset. Groupings are tagged by
assetRidanddataScopeso callers can safely compose multiple assets withcombine.
Set
typeto"run".runRunSingle run. Expands to one grouping per data source attached to the run: asset-typed sources contribute one grouping per data scope on the asset; dataset-typed sources contribute one grouping per dataset. Groupings are tagged by
runRidanddataScope; asset-typed groupings additionally carryassetRidso callers can safely compose multiple runs withcombine.
Set
typeto"saved".savedSavedDataset
Set
typeto"search".searchSearchDataset
Set
typeto"combine".combineCombinedDataset
Set
typeto"tag".tagTaggedDataset
Set
typeto"filter".filterFilteredDataset
Set
typeto"timeShift".timeShiftTimeShiftedDataset
Set
typeto"alignByAnchor".alignByAnchorAnchorAlignedDataset
Set
typeto"withSeries".withSeriesWithSeriesDataset
Set
typeto"dropSeries".dropSeriesDropSeriesDataset
Set
typeto"reference".referenceDatasetReference
Set
typeto"withDimensions".withDimensionsDimensionsDataset
commitCommit
Fields of
Commitidstring
resourceRidstring (RID)
parentCommitstring
mergeParentCommitstring
messagestring
isWorkingStatebooleanA working state commit is created via the
saveWorkingStateendpoint and is non-permanent. In the future, it may be compacted and not exist.committedBystring (RID)
committedAtstring (date-time)
channelSearchSplitTagKeysarray of stringTag keys whose distinct values channel search surfaces as separate series. Each listed key is retained on this dataset’s channel-search locators, so a channel carrying two values of that key appears once per value instead of collapsing into a single entry — the behavior a data scope has on an asset. Keys not listed here stay available for filtering and grouping but do not contribute to series cardinality. Every key must be produced by this dataset’s
derivedDefinition. Ingested tag keys cannot be split on.
Request
curl -X PUT "https://api.gov.nominal.io/api/catalog/v1/datasets/<rid>/bounds-from-streaming" \
-H "Authorization: Bearer <api key>" \
-H "Content-Type: application/json" \
--data '{
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
}
}'
import requests
response = requests.put(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<rid>/bounds-from-streaming",
headers={"Authorization": "Bearer <api key>"},
json={
"bounds": {
"start": {"seconds": 1, "nanos": 1},
"end": {"seconds": 1, "nanos": 1},
"type": "RELATIVE",
},
},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.put(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<rid>/bounds-from-streaming",
headers={"Authorization": "Bearer <api key>"},
json={
"bounds": {
"start": {"seconds": 1, "nanos": 1},
"end": {"seconds": 1, "nanos": 1},
"type": "RELATIVE",
},
},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.put(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<rid>/bounds-from-streaming",
headers={"Authorization": "Bearer <api key>"},
json={
"bounds": {
"start": {"seconds": 1, "nanos": 1},
"end": {"seconds": 1, "nanos": 1},
"type": "RELATIVE",
},
},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"rid": "<rid>",
"name": "<name>",
"handle": {
"type": "s3",
"s3": {
"bucket": "<bucket>",
"key": "<key>"
}
},
"description": "<description>",
"originMetadata": {
"path": "<path>",
"xSeriesIsAbsolute": true,
"schemaDirectivePath": "<schema directive path>",
"xSeriesColumnName": "<x series column name>",
"xSeriesTimeUnit": "DAYS",
"timestampMetadata": {
"seriesName": "<series name>",
"timestampType": {
"type": "relative",
"relative": {
"timeUnit": "DAYS",
"offset": "2026-01-01T00:00:00Z"
}
}
},
"channelConfig": {
"prefixTreeDelimiter": "<prefix tree delimiter>"
}
},
"bounds": {
"start": {
"seconds": 1,
"nanos": 1
},
"end": {
"seconds": 1,
"nanos": 1
},
"type": "RELATIVE"
},
"properties": {
"<key>": "<property value>"
},
"typedProperties": {
"<key>": {
"type": "stringValue",
"stringValue": "<property value>"
}
},
"labels": [
"<label>"
],
"timestampType": "ABSOLUTE",
"allowStreaming": true,
"granularity": "NANOSECONDS",
"isArchived": true,
"defaultGroupByAndAggregationConfig": {
"numericAggregation": {
"type": "sum",
"sum": {}
}
},
"derivedDefinition": {
"spec": {
"type": "asset",
"asset": {
"rid": {
"type": "literal",
"literal": "<literal>"
}
}
},
"commit": {
"id": "<commit id>",
"resourceRid": "<resource rid>",
"parentCommit": "<commit id>",
"mergeParentCommit": "<commit id>",
"message": "<message>",
"isWorkingState": true,
"committedBy": "<user rid>",
"committedAt": "2026-01-01T00:00:00Z"
}
},
"channelSearchSplitTagKeys": [
"<tag name>"
]
}
Archive dataset#
POST /catalog/v1/datasets/{datasetRid}/archive
Archives a dataset, which will hide it from search results unless the includeArchived flag is set to true. The dataset can still be directly accessed by its UUID/rid.
Path parameters
datasetRidstring (RID) required
Response
No content (HTTP 204).
Request
curl -X POST "https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/archive" \
-H "Authorization: Bearer <api key>"
import requests
response = requests.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/archive",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
import httpx
response = httpx.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/archive",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/archive",
headers={"Authorization": "Bearer <api key>"},
) as response:
response.raise_for_status()
asyncio.run(main())
Response: 204
No content.
Unarchive dataset#
POST /catalog/v1/datasets/{datasetRid}/unarchive
Undoes the archiving of a dataset.
Path parameters
datasetRidstring (RID) required
Response
No content (HTTP 204).
Request
curl -X POST "https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/unarchive" \
-H "Authorization: Bearer <api key>"
import requests
response = requests.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/unarchive",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
import httpx
response = httpx.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/unarchive",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.post(
"https://api.gov.nominal.io/api/catalog/v1/datasets/<dataset rid>/unarchive",
headers={"Authorization": "Bearer <api key>"},
) as response:
response.raise_for_status()
asyncio.run(main())
Response: 204
No content.
Get all properties and labels#
GET /catalog/v1/datasets/all-properties-labels
This legacy bulk-list surface returns string property values only.
Query parameters
workspacesarray of string (RID)If supplied, will return only the properties and labels that are associated with the supplied workspaces. If empty will return all properties and labels for workspaces that the user is permitted to see.
Response
propertiesmap ofPropertyNameto array of string
labelsarray of string
Request
curl "https://api.gov.nominal.io/api/catalog/v1/datasets/all-properties-labels" \
-H "Authorization: Bearer <api key>"
import requests
response = requests.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/all-properties-labels",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/all-properties-labels",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.get(
"https://api.gov.nominal.io/api/catalog/v1/datasets/all-properties-labels",
headers={"Authorization": "Bearer <api key>"},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
{
"properties": {
"<key>": [
"<property value>"
]
},
"labels": [
"<label>"
]
}
Get log dataset for workspace#
GET /catalog/v1/workspaces/{workspaceRid}/log-dataset
Returns the log dataset RID for the specified workspace if configured and accessible to the caller.
Path parameters
workspaceRidstring (RID) required
Response
The body is a string (RID).
When there is nothing to return, the response is HTTP 204 with no body.
Request
curl "https://api.gov.nominal.io/api/catalog/v1/workspaces/<workspace rid>/log-dataset" \
-H "Authorization: Bearer <api key>"
import requests
response = requests.get(
"https://api.gov.nominal.io/api/catalog/v1/workspaces/<workspace rid>/log-dataset",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import httpx
response = httpx.get(
"https://api.gov.nominal.io/api/catalog/v1/workspaces/<workspace rid>/log-dataset",
headers={"Authorization": "Bearer <api key>"},
)
response.raise_for_status()
print(response.json())
import asyncio
import aiohttp
async def main() -> None:
async with aiohttp.ClientSession() as session:
async with session.get(
"https://api.gov.nominal.io/api/catalog/v1/workspaces/<workspace rid>/log-dataset",
headers={"Authorization": "Bearer <api key>"},
) as response:
response.raise_for_status()
print(await response.json())
asyncio.run(main())
Response: 200
"<dataset rid>"
Errors#
Errors shows how these errors are sent.
Each error’s errorName is Catalog: followed by its name below.
Status |
Name |
Notes |
|---|---|---|
400 |
|
Derived datasets are backed by other datasets and do not support file ingest. |
400 |
|
The current dataset is legacy and does not support multiple files. Please re-ingest this dataset. |
400 |
|
Derived datasets cannot currently be backed by other derived datasets. |
400 |
|
Derived datasets can currently only be backed by ordinary datasets. |
400 |
|
A derived definition can only be initialized on a dataset with no ingest or streaming state: no files, no create-time file handle, no global bounds, no external connection config, and no streaming sessions. |
400 |
|
Dataset derived definitions can only be updated for datasets that are already derived. To initialize a derived definition on an empty dataset, use createDatasetDerivedDefinition. |
400 |
|
A channel-search split tag key is not injected by the dataset’s derived definition. Split tag keys must name a tag the definition produces. |
400 |
|
Channel-search split tag keys were supplied for a dataset with no derived definition. Only tag keys a derived definition produces can be split on; ingested tag keys cannot. |
400 |
|
Different time units are not allowed for the same dataset. Must be all nanoseconds. |
404 |
|
|
404 |
|
|
404 |
|
|
404 |
|
|
404 |
|
|
409 |
|
|
409 |
|
Dataset files must have globally unique S3 paths on creation. On update, the ingested_at timestamp must be unique. |
409 |
|
|
409 |
|
The dataset already has a derived definition. Use commitDerivedDefinition to update it. |
409 |
|
The dataset with the requested path already exists in another workspace. |
409 |
|
The current dataset is not in a valid state for uploading an additional file - it is either failed or still ingesting the previous file. |
500 |
|
Derived dataset creation is not enabled for this environment. |
Each error’s errorName is Scout: followed by its name below.
Status |
Name |
Notes |
|---|---|---|
404 |
|
Types#
Abs#
Computes the absolute value of each point in the series.
xNumericSeriesrequiredInput numeric series.
AbsoluteThreshold#
Threshold defined as a real number corresponding the unit of a series.
valueDoubleConstantrequiredAllowed spread, in the series’ own unit.
AbsoluteTimestamp#
Set type to the name of one of the other fields, and set only that field.
iso8601Iso8601Timestamp
epochOfTimeUnitEpochTimestamp
customFormatCustomTimestamp
AbsoluteTimestampSeries#
Replaces values in a series with the absolute timestamps.
Acos#
Computes the arccosine of each value, returning radians.
xNumericSeriesrequiredInput numeric series (values in [-1, 1]).
Add#
Adds two numeric series pointwise.
leftNumericSeriesrequiredLeft-hand operand.
rightNumericSeriesrequiredRight-hand operand.
AfterPersistenceWindow#
The first point in the output range will be the first point after the condition has been true for the min duration and min points.
An empty JSON object, {}.
AggregateEnumSeries#
Aggregates values with duplicate timestamps in the input series values into a single value using the specified aggregation function.
inputEnumSeriesrequired
functionEnumAggregationFunctionrequired
AggregateNumericSeries#
DEPRECATED. Use a dedicated group-by-and-aggregate nodes instead.
Aggregates values with duplicate timestamps in the input series values into a single value using the specified aggregation function.
inputNumericSeriesrequired
functionNumericAggregationFunctionrequired
groupByTagsarray ofStringConstantTags to group by for the aggregation. If left empty, the tags to group by will be equivalent to those in the input series. If specified, the result will be grouped ONLY by the specified tags. The tags specified here MUST be a (non-strict) subset of the input series’s group by tags.
aggregateByAllGroupingsbooleanThis field’s purpose is to distinguish between the two flavors of groupByTags being empty. It has no effect when groupByTags is non-empty. When true and groupByTags is empty, aggregate across all series in the input. When false and groupByTags is empty, the result will be grouped by the same tags as the input series.
interpolationConfigurationInterpolationConfigurationdeprecatedIf provided, interpolates values at timestamps where the input series has values before aggregating. If not provided, only aggregates when timestamps match exactly (existing behavior). Deprecated: Setting interpolation on this node is deprecated.
AggregateUnderRangesSeries#
Aggregates the input series under the ranges in the provided ranges series, and outputs a new series with one point at the start of each range, where the value is the aggregation result. If a range has no points, no point will be produced for it. If a range has no start, the point produced for it will be at the start of the compute request range.
inputNumericSeriesrequired
rangesRangeSeriesrequired
operationNumericAggregationFunctionrequired deprecatedDeprecated: Use a dedicated group-by-and-aggregate node instead (coming in a follow-up PR). This field is retained as required for backwards compatibility with existing callers.
AggregationBuilder#
Set type to the name of one of the other fields, and set only that field.
rollingTimeRollingTimeAggregationBuilder
rollingPointsRollingPointsAggregationBuilder
groupByGroupByAggregationBuilder
Alignment#
Specifies how to set the timestamp grid for multi-input operations.
Set type to the name of one of the other fields, and set only that field.
driverSeriesDriverSeries
unionUnion
AllChannelsConnectionsScrapingConfig#
An empty JSON object, {}.
AllowNegativeValues#
An empty JSON object, {}.
AnchorAlignedDataset#
Aligns each anchored portion of the input dataset onto a common anchor timestamp while preserving grouping tags. Without a reference the selected anchor becomes timestamp zero, so the output is in a relative time domain; with a reference the anchors are aligned onto the referenced portion’s anchor and the output stays in the input’s absolute time domain. Currently run boundary, event, and range anchors are supported, and every storage-backed leaf of the input must resolve to run-backed branches.
inputapi.DatasetrequiredThe dataset to align.
anchorDatasetAnchorrequiredSelects the timestamp each anchored portion of the input is aligned by.
referenceAnchorReferenceWhen present, each anchor is aligned onto the referenced anchor timestamp instead of onto timestamp zero, so output remains in the input’s absolute time domain and the referenced portion is unshifted. When absent, the selected anchor becomes timestamp zero.
AnchorReference#
Selects the anchor timestamp every other anchor is aligned onto.
Set type to the name of one of the other fields, and set only that field.
earliestAnchorEarliestAnchorReferenceAligns every anchor onto the earliest anchor resolved from the input, so the earliest-anchored run is unshifted and every other run shifts backwards onto it.
AndSeries#
Combines two boolean series point-wise using logical AND, producing true where both inputs are true and false otherwise.
leftBooleanSeriesrequired
rightBooleanSeriesrequired
api.Dataset#
A logical bundle of named series. For the time being, columns are each a time series (series name + tags). A dataset is either rooted in persisted data (asset or dataset) or defined as a transformation of another dataset.
Set type to the name of one of the other fields, and set only that field.
assetAssetSingle asset. Expands to one grouping per data scope on the asset. Groupings are tagged by
assetRidanddataScopeso callers can safely compose multiple assets withcombine.runRunSingle run. Expands to one grouping per data source attached to the run: asset-typed sources contribute one grouping per data scope on the asset; dataset-typed sources contribute one grouping per dataset. Groupings are tagged by
runRidanddataScope; asset-typed groupings additionally carryassetRidso callers can safely compose multiple runs withcombine.savedSavedDataset
searchSearchDataset
combineCombinedDataset
tagTaggedDataset
filterFilteredDataset
timeShiftTimeShiftedDataset
alignByAnchorAnchorAlignedDataset
withSeriesWithSeriesDataset
dropSeriesDropSeriesDataset
referenceDatasetReference
withDimensionsDimensionsDataset
ApiScrapingConfig#
This config is used to scrape data from an external connections service.
Set type to the name of one of the other fields, and set only that field.
allChannelsAllChannelsConnectionsScrapingConfig
channelAllowListChannelAllowListConnectionsScrapingConfig
channelBlockListChannelBlockListConnectionsScrapingConfig
ApproximateThresholdOperator#
EQUAL_TO
NOT_EQUAL_TO
ApproximateThresholdRanges#
Produces a list of ranges for which the threshold condition is satisfied.
inputNumericSeriesrequired
thresholdDoubleConstantrequired
toleranceDoubleConstantrequired
operatorApproximateThresholdOperatorrequired
persistenceWindowConfigurationPersistenceWindowConfiguration
ArithmeticSeries#
inputsmap ofLocalVariableNametoNumericSeries
expressionstring required
interpolationConfigurationInterpolationConfigurationdeprecatedDefaults to forward fill interpolation with a 1s interpolation radius Deprecated: Setting interpolation on this node is deprecated.
ArraySeries#
Set type to the name of one of the other fields, and set only that field.
numeric1dNumeric1dArraySeries
enum1dEnum1dArraySeries
Asin#
Computes the arcsine of each value, returning radians.
xNumericSeriesrequiredInput numeric series (values in [-1, 1]).
Asset#
Asset reference for Dataset.asset. The rid field accepts either a literal asset RID or a StringConstant reference resolved from Context.
ridStringConstantrequiredResource identifier of an asset
AssetChannel#
assetRidStringConstantrequired
dataScopeNameStringConstantrequiredUsed to disambiguate when multiple data scopes within this asset contain channels with the same name.
channelStringConstantrequired
additionalTagsmap of string toStringConstantdeprecatedDeprecated: Use additionalTagFilters
additionalTagFiltersTagFiltersTags to filter the channel by, in addition to tag filters defined for a given Asset data scope. Throws on collisions with tag keys already defined for the given Asset data scope. Only returns points that match both sets of tag filters. For log series, include arg filters here in addition to tag filters.
tagsToGroupByarray of string deprecatedDeprecated: Use groupByTags instead. Throws if both tagsToGroupBy and groupByTags are specified.
groupByTagsarray ofStringConstantTags that the channel should be grouped by. If this is non-empty a grouped result will be returned with an entry for each grouping. Only one of tagsToGroupBy and groupByTags should be specified.
AssetsSearchTarget#
Search assets. Each match expands to one grouping per data scope on that asset.
queryResourceSearchQueryrequiredPredicate tree used to filter matching assets.
Atan2#
Computes the two-argument arctangent atan2(y, x) using signs to determine the quadrant.
yNumericSeriesrequiredY-coordinate series.
xNumericSeriesrequiredX-coordinate series.
Average#
A average that gives the same weight to each observation inside the time window
An empty JSON object, {}.
BandPassConfiguration#
lowPassbandFrequencyDoubleConstantrequired
highPassbandFrequencyDoubleConstantrequired
BandStopConfiguration#
lowStopbandFrequencyDoubleConstantrequired
highStopbandFrequencyDoubleConstantrequired
BigQueryChannelNameComponent#
includeProjectboolean requiredWhether to include the project name in the channel name.
includeDatasetboolean requiredWhether to include the dataset name in the channel name.
includeTableboolean requiredWhether to include the table name in the channel name.
BigQueryConnectionDetails#
regionstring requiredThe region of the BigQuery Project (e.g. “us-east1”)
projectstring requiredThe name of the BigQuery Project
datasetstring requiredThe name of the dataset within the project
tablestring requiredThe name of the table within the dataset
serviceAccountKeySecretRidstring (RID) requiredSecret Rid of service account key stored in Secrets Service.
BigQueryScrapingConfig#
timeColumnstring requiredThe name of the column that holds the timestamp.
tagColumnsarray of stringThe name of the columns that should be interpreted as tag columns
channelNameComponentsBigQueryChannelNameComponentchannelNameComponents will be combined, together with separator, to form a fully qualified channel name. By default, we don’t add anything to the column name.
separatorstringThe separator that delimits the parts of the channel name. If ommitted, the default is a “.”.
BinaryArithmeticOperation#
ATAN2Calculates the angle measure between the x-axis and a ray from the origin to a point (input2, input1) in radians. Note the order of arguments.
BinaryArithmeticSeries#
Applies a point-wise transformation to a pair of series.
input1NumericSeriesrequired
input2NumericSeriesrequired
operationBinaryArithmeticOperationrequired
interpolationConfigurationInterpolationConfigurationdeprecatedDefaults to forward fill interpolation with a 1s interpolation radius Deprecated: Setting interpolation on this node is deprecated.
BitAnd#
Applies a bitwise AND between each value (cast to int64) and the operand.
inputNumericSeriesrequiredInput numeric series (values cast to int64).
operandIntegerConstantrequiredThe value to AND with each point.
BitAndFunction#
operandinteger (int64) required
BitOperationFunction#
Set type to the name of one of the other fields, and set only that field.
andBitAndFunction
orBitOrFunction
xorBitXorFunction
shiftRightBitShiftRightFunction
shiftLeftBitShiftLeftFunction
bitTestBitTestFunction
BitOperationSeries#
Casts input series values to long before applying the bitwise operation.
inputNumericSeriesrequired
functionBitOperationFunctionrequired
BitOr#
Applies a bitwise OR between each value (cast to int64) and the operand.
inputNumericSeriesrequiredInput numeric series (values cast to int64).
operandIntegerConstantrequiredThe value to OR with each point.
BitOrFunction#
operandinteger (int64) required
BitShiftLeft#
Left-shifts the bits in each value (cast to int64) by the given number of positions.
inputNumericSeriesrequiredInput numeric series (values cast to int64).
operandIntegerConstantrequiredNumber of positions to shift left.
BitShiftLeftFunction#
Shifts the bits in each value left according to the given operand.
operandinteger required
BitShiftRight#
Right-shifts the bits in each value (cast to int64) by the given number of positions.
inputNumericSeriesrequiredInput numeric series (values cast to int64).
operandIntegerConstantrequiredNumber of positions to shift right.
BitShiftRightFunction#
Right shifts the bits in each value right according to the given operand.
operandinteger required
BitTest#
Returns the bit at the specified index for each value (cast to int64), where index 0 is the least significant bit.
inputNumericSeriesrequiredInput numeric series (values cast to int64).
indexIntegerConstantrequiredZero-based bit index, where 0 is the least significant bit.
BitTestFunction#
Returns the bit at the specified index, where the right-most bit has index 0.
indexinteger required
BitXor#
Applies a bitwise XOR between each value (cast to int64) and the operand.
inputNumericSeriesrequiredInput numeric series (values cast to int64).
operandIntegerConstantrequiredThe value to XOR with each point.
BitXorFunction#
operandinteger (int64) required
BooleanRanges#
Produces ranges spanning the regions where a boolean input is true.
The condition is evaluated point-wise at each aligned timestamp; the numeric series it compares are aligned against each other the same way arithmetic between them would be.
inputBooleanSeriesrequiredThe condition whose true regions become ranges.
minPointsIntegerConstantMinimum number of matching points. Defaults to 1.
minDurationcompute.api.DurationMinimum matching duration. Defaults to one nanosecond.
rangeStartRangeStartSelects whether the output begins at the first match or after persistence is satisfied.
BooleanSeries#
Set type to the name of one of the other fields, and set only that field.
greaterThanGreaterThanSeries
lessThanLessThanSeries
equalToEqualToSeries
notEqualToNotEqualToSeries
greaterThanOrEqualToGreaterThanOrEqualToSeries
lessThanOrEqualToLessThanOrEqualToSeries
andAndSeries
orOrSeries
Bounds#
startTimestamprequired
endTimestamprequired
typedatasource.TimestampTyperequired
BuiltinConstant#
A system-defined constant referenceable by name. The server resolves the symbol to its numeric value during query resolution.
Set type to the name of one of the other fields, and set only that field.
mathMathConstant
physicalPhysicalConstant
BuiltInCurveFit#
A closed-form curve fit (exponential, logarithmic, or power) of the enclosing target (y) against the chosen independent variable (x): either sample time (a time variable) or another series (a numeric variable).
independentVariableCurveFitVariablerequiredThe
xaxis the target is fit against — either relative sample time or another series.
catalog.Dataset#
ridstring (RID) required
namestring required
handleHandledeprecatedDeprecated: Deprecated. Use DatasetFile#handle
descriptionstring
originMetadataDatasetOriginMetadatarequired
boundsBounds
propertiesmap ofPropertyNameto string deprecatedDeprecated: use typedProperties
typedPropertiesmap ofPropertyNametoTypedPropertyValue
labelsarray of string
timestampTypeWeakTimestampTyperequired
allowStreamingboolean required
granularityGranularityrequired
isArchivedboolean required
defaultGroupByAndAggregationConfigDefaultGroupByAndAggregationConfig
derivedDefinitionDerivedDefinitionDefinition for a derived dataset. When present, this dataset is virtual and backed by the referenced provided Dataset definition instead of persisted dataset files.
channelSearchSplitTagKeysarray of stringTag keys whose distinct values channel search surfaces as separate series. Each listed key is retained on this dataset’s channel-search locators, so a channel carrying two values of that key appears once per value instead of collapsing into a single entry — the behavior a data scope has on an asset. Keys not listed here stay available for filtering and grouping but do not contribute to series cardinality. Every key must be produced by this dataset’s
derivedDefinition. Ingested tag keys cannot be split on.
catalog.TimestampType#
Set type to the name of one of the other fields, and set only that field.
relativeRelativeTimestamp
absoluteAbsoluteTimestamp
CategoricalEqualityRangesV2#
Produces ranges where all categorical inputs are equal or are not all equal.
inputsarray ofEnumSeriesThe series compared against each other. At least two are required.
minPointsIntegerConstantMinimum number of matching points. Defaults to 1.
minDurationcompute.api.DurationMinimum matching duration. Defaults to one nanosecond.
rangeStartRangeStartSelects whether the output begins at the first match or after persistence is satisfied.
equalityOperatorEqualityOperatorrequired
CategoricalOnChangeRangesV2#
Produces a zero-duration range whenever a categorical input changes value.
inputEnumSeriesrequired
CategoricalStaleRangesV2#
Produces ranges where a categorical input has no data for at least a duration.
inputEnumSeriesrequired
thresholdcompute.api.DurationrequiredMinimum gap without data that produces an output range.
CategoryFilterRangesV2#
Produces ranges where a categorical input matches a set of values.
inputEnumSeriesrequired
valuesarray ofStringConstantCategory labels matched against each input point.
minPointsIntegerConstantMinimum number of matching points. Defaults to 1.
minDurationcompute.api.DurationMinimum matching duration. Defaults to one nanosecond.
rangeStartRangeStartSelects whether the output begins at the first match or after persistence is satisfied.
operatorEnumFilterOperatorrequired
ChannelAllowListConnectionsScrapingConfig#
channelAllowListarray of stringOnly channels in this set will be scraped. Must not be empty.
ChannelBlockListConnectionsScrapingConfig#
channelBlockListarray of stringOnly channels not in this set will be scraped. Must not be empty.
ChannelConfig#
prefixTreeDelimiterstringIf set, will construct a prefix tree for channels of the dataset using the given delimiter.
ChannelSeries#
Deprecated: prefer SelectSeries-based nodes with an explicit dataset (or VariableValue.series when binding a series to a variable), which select from an arbitrary dataset and let the consuming node decide the series type.
Set type to the name of one of the other fields, and set only that field.
dataSourceDataSourceChannel
assetAssetChannel
runRunChannel
CombinedDataset#
Combines multiple datasets into a single dataset by taking the union of all points from each input (akin to SQL {@code UNION ALL}). Duplicate timestamps are preserved rather than merged; callers that need aggregation should wrap the selected series downstream.
inputsarray ofapi.DatasetThe datasets to combine.
CombineStructSeries#
Combines named input series into one struct-valued series. Each map key becomes a field name in the resulting struct; a downstream UDF receives the struct as a JSON object keyed by these names and parses it itself. Alignment uses the interpolation configuration before producing row-aligned values.
inputsmap of string toSeriesInput series keyed by struct field name.
interpolationConfigurationInterpolationConfigurationdeprecatedOptional interpolation rules used when aligning input series before combining them. Deprecated: Setting interpolation on this node is deprecated.
Commit#
idstring required
resourceRidstring (RID) required
parentCommitstring
mergeParentCommitstring
messagestring required
isWorkingStateboolean requiredA working state commit is created via the
saveWorkingStateendpoint and is non-permanent. In the future, it may be compacted and not exist.committedBystring (RID) required
committedAtstring (date-time) required
compute.api.Duration#
A length of time. Can be constructed from literal values in various units or composed via arithmetic. Used for window sizes, time shifts, persistence thresholds, and resampling intervals.
Set type to the name of one of the other fields, and set only that field.
referenceReference
nanosecondsDurationNanoseconds
millisecondsDurationMilliseconds
secondsDurationSeconds
minutesDurationMinutes
hoursDurationHours
daysDurationDays
addDurationAdd
subtractDurationSubtract
negateDurationNegate
ComputeEventQuery#
Set type to the name of one of the other fields, and set only that field.
searchTextStringConstant
afterTimestampConstantFilters to events after this timestamp, exclusive. This includes events that start before, but end after this time.
beforeTimestampConstantFilters to events before this timestamp, exclusive. This includes events that start before, but end after this time.
advancedTimeFilterEventTimeFilter
assetStringConstant
templateStringConstant
workbookStringConstant
dataReviewStringConstant
originTypeSearchEventOriginType
dataReviewCheckStringConstant
dispositionStatusEventDispositionStatus
priorityStringConstant
assigneeStringConstant
eventTypeEventType
createdByStringConstant
labelStringConstant
propertyrids.api.Property
andarray ofComputeEventQuery
orarray ofComputeEventQuery
notComputeEventQuery
workspaceStringConstant
procedureStringConstant
procedureExecutionStringConstant
stepIdStringConstant
ComputeNode#
Set type to the name of one of the other fields, and set only that field.
enumEnumSeries
numericNumericSeries
logLogSeries
rangesRangeSeries
arrayArraySeries
structStructSeries
curveFitCurveFitDeprecated: use
curveV2(CurveFitV2) instead.curveV2CurveFitV2
videoReference
rawReferencedeprecatedDeprecated: wrap raw references in a typed compute node.
ComputeNodeWithContext#
seriesNodeComputeNoderequired
contextContextrequired
connection.api.Duration#
unitTemporalUnitrequired
countinteger required
ConnectionDetails#
Set type to the name of one of the other fields, and set only that field.
timescaleTimescaleConnectionDetails
influxInflux2ConnectionDetails
influx1Influx1ConnectionDetails
nominalNominalConnectionDetails
timestreamTimestreamConnectionDetails
visualCrossingVisualCrossingConnectionDetails
bigQueryBigQueryConnectionDetails
ConnectionStatus#
isConnectedboolean required
ConstantNumericSeries#
A synthetic series that produces a constant value across the entire time range.
Set type to the name of one of the other fields, and set only that field.
doubleDoubleConstantCreate from a floating-point constant.
ContainsSeries#
Maps each log line to 1.0 when operator matches it and 0.0 otherwise.
inputLogSeriesrequired
operatorLogFilterOperatorrequired
Context#
variablesmap ofVariableNametoVariableValue
functionVariablesmap ofFunctionReferencetoFunctionVariablesdeprecatedDeprecated: This field is deprecated and will be removed in a future version.
datasetReferencesmap ofDatasetReferenceNametoapi.DatasetNamed datasets looked up by
DatasetReferencenodes in the compute tree.defaultFillStrategyFillStrategyDefault fill strategy applied at every multi-input alignment boundary where no per-node interpolationConfiguration is specified. When absent, falls back to the server-configured default (forward fill with a 1 second fill limit).
defaultAlignmentStrategyAlignmentDefault alignment strategy applied at every multi-input alignment boundary where no per-node alignment strategy is specified. When absent, falls back to the server default (driver-series alignment).
applyDataScopeOffsetsbooleanWhen true, dataset resolution applies each data scope’s persisted offset as a time shift, and the client must not pre-apply those offsets itself. When absent or false, scope offsets are left to the client, preserving behavior for legacy clients that inject equivalent timeShift nodes into the compute request. Read from the context in scope at resolution time.
resolveAssetsThroughDatasetsbooleanWhen true,
Dataset.asset, the asset-typed sources ofDataset.run, and asset or run search expand through each asset’s mirrored derived dataset instead of its data scopes. Branches then carry the dataset’s declared channel-search split tag keys in place of the reservedDATA_SCOPEtag, so callers must filter and group on those keys; the definition’s own time shifts apply regardless ofapplyDataScopeOffsets; and an asset with no mirrored dataset resolves to no data. When absent or false, assets expand from their data scopes as before.
Cos#
Computes the trigonometric cosine of each value (input in radians).
xNumericSeriesrequiredInput numeric series (radians).
Count#
The number of points inside the time window.
An empty JSON object, {}.
CreateDataset#
namestring required
handleHandledeprecatedDeprecated: Handles should only be specified when adding files to datasets
metadatamap of string to string deprecatedDeprecated: Deprecated in favor of properties
originMetadataDatasetOriginMetadatarequired
labelsarray of string
propertiesmap ofPropertyNameto string deprecatedUser-defined dataset properties. The UTF-8 JSON serialization must not exceed 2,000 bytes. Deprecated: use typedProperties
typedPropertiesmap ofPropertyNametoTypedPropertyValue
descriptionstring
granularityGranularityGranularity of dataset timestamps. Defaults to nanoseconds.
isV2Datasetboolean deprecatedIf true, the dataset should be ingested to the v2 tables and is compatible with streaming. Deprecated: Regardless of what value is specified, this flag is treated as true by the service.
workspacestring (RID)The workspace in which to create the dataset. If not provided, the dataset will be created in the default workspace for the user’s organization, if the default workspace for the organization is configured.
markingRidsarray of string (RID)The markings to apply to the created dataset. If not provided, the dataset will be visible to all users in the same workspace.
derivedDefinitionCreateDerivedDefinitionDefinition for a derived dataset. When present, the created dataset is virtual and does not ingest or persist dataset files.
channelSearchSplitTagKeysarray of stringTag keys whose distinct values channel search surfaces as separate series. Every key must be produced by
derivedDefinition; ingested tag keys, and keys supplied without aderivedDefinition, are rejected.defaultGroupByAndAggregationConfigDefaultGroupByAndAggregationConfigIf set, the dataset’s default group-by and aggregation config.
CreateDerivedDefinition#
specapi.DatasetrequiredInitial derived dataset logic.
messagestring requiredInitial commit message for the derived definition’s main branch.
CumulativeSumSeries#
Calculates the running total of the series values.
inputNumericSeriesrequired
startTimestampTimestampConstantrequired
CurveFit#
Deprecated: use CurveFitV2 instead.
curveFitPlotTypeCurveFitPlotTyperequired
curveFitDetailsCurveFitDetailsrequired
curveFitOptionsCurveFitOptionsrequired
CurveFitBoundsConfiguration#
minDoubleConstant
maxDoubleConstant
CurveFitDetails#
Set type to the name of one of the other fields, and set only that field.
exponentialExponentialCurve
logarithmicLogarithmicCurve
polynomialPolynomialCurve
powerPowerCurve
CurveFitOptions#
startTimeTimestampConstantrequired
endTimeTimestampConstantrequired
CurveFitPlotType#
Set type to the name of one of the other fields, and set only that field.
timeSeriesTimeSeriesCurveFit
scatterScatterCurveFit
CurveFitRange#
startTimeTimestampConstantrequired
endTimeTimestampConstantrequired
CurveFitSeriesVariable#
seriesNumericSeriesrequired
boundsConfigCurveFitBoundsConfigurationOptional inclusive bounds selecting which variable samples are fit. Samples outside the bounds are excluded.
CurveFitTarget#
The series a curve is fit against (the y of the fit). Modeled as a union so additional target types can be added in the future; today the only arm is a numeric series.
Set type to the name of one of the other fields, and set only that field.
numericCurveFitSeriesVariable
CurveFitTimeVariable#
timeUnitTimeUnitThe unit of the time variable in the formula expression. If not specified, the time unit is assumed to be seconds.
CurveFitV2#
targetCurveFitTargetrequiredTarget to fit the equation against.
fitRangeCurveFitRangerequiredThe range to fit the curve over
curveToFitCurveFitV2DetailsrequiredThe curve type to fit against the target series.
CurveFitV2Details#
Set type to the name of one of the other fields, and set only that field.
exponentialBuiltInCurveFit
logarithmicBuiltInCurveFit
polynomialPolynomialCurveFit
powerBuiltInCurveFit
customCustomCurveFit
CurveFitVariable#
A series a curve fit reads from: another numeric series, or the target’s own sample timestamps. Used both as the named formula inputs of a custom fit and as the independent (x) axis of a built-in fit.
Set type to the name of one of the other fields, and set only that field.
numericCurveFitSeriesVariable
timeCurveFitTimeVariable
CustomCurveFit#
formulaStringConstantrequiredA mathematical expression in terms of variable and parameter names, e.g.
a * x + b * sin(x). Variable names must match keys in thevariablesmap; parameter names must match keys in theparametersmap.variablesmap ofLocalVariableNametoCurveFitVariableNamed input series referenced in the formula. Keys must match the variable names used in the formula expression.
parametersmap of string toDoubleConstantNamed fitting parameters and their initial guesses. Keys must match the parameter names used in the formula expression; values are the starting points passed to the optimizer. The target the curve is fit against is the
yof the enclosing curve fit (the timeSeries series values, or the scatteryseries).
CustomTimestamp#
formatstring requiredThe format string should be in the format of the
DateTimeFormatterclass in Java.defaultYearintegerYear is accepted as an optional field for cases like IRIG time format, and will be assumed as current year if not provided.
defaultDayOfYearintegerDefault day of year is accepted as an optional field for cases like IRIG time format and will be overridden by day of year in time format.
DatasetAnchor#
Set type to the name of one of the other fields, and set only that field.
runIntervalAnchorUses the start or end of each run as its anchor. The input dataset must resolve exclusively to run-backed branches.
DatasetFile#
idstring (UUID) required
datasetRidstring (RID) required
namestring required
handleHandlerequired
fileSizeBytesinteger (int64)Size in bytes of the stored dataset file; for transforming ingests this is the produced output, not the original upload.
boundsBounds
uploadedAtstring (date-time) requiredTimestamp that the file was received and stored, but not processed or made available to consumers.
ingestedAtstring (date-time)Timestamp that the file is ingested at and made available for processing. If the file has failed to be ingested for any reason or is still being processed, then this value will be empty.
ingestStatusIngestStatusV2requiredIngest status. The in-progress sub-phases (queued, parsing, ingesting) are surfaced as inProgress with the phase carried in InProgressResult.inProgressDetails, so consumers see a stable top-level lifecycle.
timestampMetadataTimestampMetadata
ingestTagMetadataIngestTagMetadata
originFilePathsarray of string
ingestJobRidstring (RID)
ingestTransformRidstring (RID)The ingest transform within ingestJobRid that produced this file. An ingest job may fan out to many transforms, so this pins the file to the specific unit of work (and extractor) that created it — resolvable via IngestJobService#getIngestTransform. Extractor logs are tagged with this transform’s UUID (the RID locator), so this also links a file to the logs of the transform that produced it. Absent for directly uploaded files and for files ingested before this link was recorded.
deletedAtstring (date-time)Timestamp that the file is deleted at, only present if the file has been deleted.
metadataDatasetFileMetadataFile-type-specific metadata. For video files, contains timestamp manifest and segment metadata.
DatasetFileMetadata#
Metadata specific to different file types stored in datasets.
Set type to the name of one of the other fields, and set only that field.
videoVideoFileMetadata
DatasetFileSortField#
UPLOADED_AT
DatasetFileSortOptions#
isDescendingboolean required
fieldDatasetFileSortFieldrequired
DatasetOriginMetadata#
pathstringDeprecated in favor of FileOriginMetadata
xSeriesIsAbsolutebooleanDeprecated in favor of FileOriginMetadata
schemaDirectivePathstringDeprecated in favor of FileOriginMetadata
xSeriesColumnNamestringDeprecated in favor of FileOriginMetadata
xSeriesTimeUnitTimeUnitDeprecated in favor of FileOriginMetadata
timestampMetadataTimestampMetadataDeprecated in favor of FileOriginMetadata
channelConfigChannelConfig
DatasetReference#
References a named dataset in Context.datasetReferences.
nameStringConstantrequiredName of the dataset reference to resolve from the context.
datasource.TimestampType#
RELATIVE
ABSOLUTE
DataSourceChannel#
dataSourceRidStringConstantrequired
channelStringConstantrequired
tagsmap of string toStringConstantdeprecatedDeprecated: Use tagFilters
tagFiltersTagFiltersTags to filter the channel by. Only returns points from the channel where tag values match the provided expression. For log series, include arg filters here in addition to tag filters.
tagsToGroupByarray of string deprecatedDeprecated: Use groupByTags instead. Throws if both tagsToGroupBy and groupByTags are specified.
groupByTagsarray ofStringConstantTags that the channel should be grouped by. If this is non-empty a grouped result will be returned with an entry for each grouping. Only one of tagsToGroupBy and groupByTags should be specified.
DefaultGroupByAndAggregationConfig#
Dataset-level defaults that clients apply when creating new channel references.
numericAggregationNumericAggregationOperatorIf set, the default aggregation applied to merge points of a numeric series that share an exact timestamp.
Deleted#
An empty JSON object, {}.
DeletionInProgress#
An empty JSON object, {}.
DerivativeSeries#
Calculates the rate of change between subsequent points.
inputNumericSeriesrequired
timeUnitTimeUnitTime unit used to calculate the derivative. Defaults to seconds if not specified.
negativeValuesConfigurationNegativeValueConfigurationDefines the strategy for handling negative output values. Defaults to allowNegativeValues if not specified.
DerivedDefinition#
A derived dataset definition pinned to the commit that produced it.
specapi.Datasetrequired
commitCommitrequired
DerivedSeries#
Represents a derived series within a compute graph. This is a series that is derived from a function within a module.
Set type to the name of one of the other fields, and set only that field.
functionFunctionDerivedSeries
Dimension#
A value derived per asset or run, injected as a tag.
tagKeyStringConstantThe tag key to inject the value under. Defaults to the key derived by
value.valueDimensionValuerequiredThe value to inject.
DimensionsDataset#
Injects one tag per dimension onto every series of the underlying dataset, valued from the asset or run the series belongs to. This makes resource metadata usable as a grouping, filtering and aggregation dimension with selectTags, tag predicates and groupByTags. Group by the dimension together with ASSET_RID (or RUN_RID) to keep each resource separate, or by the dimension alone to pivot so that resources sharing a value fall into one group.
Dimensions are only supported on asset and run branches. Any other branch, such as a saved dataset or a derived series from withSeries, gets the reserved value NOMINAL_ABSENT for every dimension. To group derived series by a dimension, apply withDimensions to the withSeries input instead, so that the series they are computed from carry the dimension tags. A dimension tag replaces a series tag with the same key.
inputapi.DatasetrequiredThe underlying dataset to tag.
dimensionsarray ofDimensionThe dimensions to inject. Must not be empty, must have at most 10 entries, and every dimension must have a distinct tag key. A tag key must not be a
ReservedTagKey, start withEVENT_LABEL_PREFIX:, or start withNOMINAL_.
DimensionValue#
Set type to the name of one of the other fields, and set only that field.
stringPropertyStringPropertyDimensionThe value of a string property on the asset or run.
identityIdentityDimensionThe value of an asset’s primary key under one of its types.
Divide#
Divides left by right pointwise.
leftNumericSeriesrequiredNumerator.
rightNumericSeriesrequiredDenominator.
DoubleConstant#
A floating-point constant that can be a literal value, a named variable reference, or a named builtin constant.
Set type to the name of one of the other fields, and set only that field.
literalnumberCreate from a literal floating-point value.
variablestringCreate from a named reference name.
builtinBuiltinConstantCreate from a named builtin constant; the server resolves the numeric value.
DriverSeries#
Uses the driver series timestamps as the alignment grid.
An empty JSON object, {}.
DropSeriesDataset#
Removes series from the base dataset by name: raw and derived series alike are no longer selectable under those names. Dropping a name absent from input has no effect. Combined with WithSeriesDataset, which can expose a series under another name, this renames a series: dropSeries(withSeries(input, [newName = select(oldName)]), [oldName]).
inputapi.DatasetrequiredThe base dataset to drop series from.
namesarray ofStringConstantNames of the series to drop.
DurationAdd#
The sum of two durations.
leftcompute.api.DurationrequiredFirst duration.
rightcompute.api.DurationrequiredSecond duration.
DurationConstant#
A duration constant that can be a literal value or a named variable reference.
Set type to the name of one of the other fields, and set only that field.
literalrun.api.DurationCreate from a literal duration value.
variablestringCreate from a named reference name.
DurationDays#
Constructs a duration from a number of days.
daysIntegerConstantrequiredThe number of days.
DurationFilterRanges#
Filters a list of ranges down to only those satisfying a threshold condition on the range’s duration.
inputRangeSeriesrequired
thresholdDurationConstantrequired
operatorThresholdOperatorrequired
unboundedBehaviorUnboundedBehavior
DurationFilterRangesV2#
Filters ranges by duration using an authored Duration expression.
inputRangeSeriesrequired
durationcompute.api.DurationrequiredThe duration each input range is compared against.
unboundedBehaviorUnboundedBehaviorHow a range with an open start or end is measured. Defaults to treating the missing bound as the edge of the compute request window.
comparisonThresholdOperatorrequired
DurationHours#
Constructs a duration from a number of hours.
hoursIntegerConstantrequiredThe number of hours.
DurationMilliseconds#
Constructs a duration from a number of milliseconds.
millisecondsIntegerConstantrequiredThe number of milliseconds.
DurationMinutes#
Constructs a duration from a number of minutes.
minutesIntegerConstantrequiredThe number of minutes.
DurationNanoseconds#
Constructs a duration from a number of nanoseconds. Limited to values within the 32-bit integer range (~2.1 billion ns, or ~2.1 seconds). For larger durations, use DurationSeconds or DurationMilliseconds.
nanosecondsIntegerConstantrequiredThe number of nanoseconds.
DurationNegate#
Negates a duration (reverses its sign).
inputcompute.api.DurationrequiredThe duration to negate.
DurationSeconds#
Constructs a duration from a number of seconds.
secondsIntegerConstantrequiredThe number of seconds.
DurationSubtract#
The difference of two durations (left minus right).
leftcompute.api.DurationrequiredDuration to subtract from.
rightcompute.api.DurationrequiredDuration to subtract.
EarliestAnchorReference#
Selects the earliest anchor resolved from the input.
An empty JSON object, {}.
Empty#
An empty JSON object, {}.
EnrichedDataset#
ridstring (RID) required
uuidstring (UUID) required
namestring required
descriptionstring
displayNamestring required
metadatamap of string to string deprecatedDeprecated: Deprecated in favor of properties.
handleHandledeprecatedDeprecated: Deprecated. Use DatasetFile#handle
ingestDatestring (date-time) requiredDataset creation time. The name is historical; use lastIngestedAt for when data was last added.
lastIngestedAtstring (date-time)When data was last added to this dataset, as the latest of a successful file ingest and streaming-session activity. Empty when the dataset has never received either.
ingestStatusIngestStatusdeprecatedDeprecated: Deprecated. Use lastIngestStatus.
originMetadataDatasetOriginMetadatarequired
lastIngestStatusIngestStatusV2required
retentionPolicyRetentionPolicyrequired
sourcestring
boundsBounds
timestampTypeWeakTimestampTyperequired
labelsarray of string
propertiesmap ofPropertyNameto string deprecatedDeprecated: use typedProperties
typedPropertiesmap ofPropertyNametoTypedPropertyValue
granularityGranularityrequired
allowStreamingboolean required
isArchivedboolean required
defaultGroupByAndAggregationConfigDefaultGroupByAndAggregationConfig
migratedFromConnectionRidstring (RID)Connection RID whose data source was migrated into this dataset. Empty for datasets that were not created by the legacy connection-to-dataset migrations, or when the original connection row no longer exists.
externalConnectionMetadataExternalConnectionMetadata
derivedDefinitionDerivedDefinitionDefinition for a derived dataset. When present, this dataset is virtual and backed by the referenced provided Dataset definition instead of persisted dataset files.
channelSearchSplitTagKeysarray of stringTag keys whose distinct values channel search surfaces as separate series. Each listed key is retained on this dataset’s channel-search locators, so a channel carrying two values of that key appears once per value instead of collapsing into a single entry — the behavior a data scope has on an asset. Keys not listed here stay available for filtering and grouping but do not contribute to series cardinality. Every key must be produced by this dataset’s
derivedDefinition. Ingested tag keys cannot be split on.
Enum1dArraySeries#
Set type to the name of one of the other fields, and set only that field.
channelChannelSeries
rawReference
derivedDerivedSeries
selectEnum1dArraySelectSeries
extractFromStructExtractEnum1dArrayFromStructSeries
toStartOfIntervalEnum1dArrayToStartOfIntervalSeries
timeShiftEnum1dArrayTimeShiftSeries
filterByTagEnum1dArrayTagFilterSeries
selectTagsEnum1dArraySelectTagsSeries
Enum1dArrayTagFilterSeries#
Applies a tag predicate to the enum 1d array series produced by input.
inputEnum1dArraySeriesrequired
predicateTagPredicaterequired
Enum1dArrayTimeShiftSeries#
inputEnum1dArraySeriesrequired
durationDurationConstantrequired
Enum1dArrayToStartOfIntervalSeries#
Snaps each point’s timestamp to the start of its enclosing fixed-size time bucket. Buckets are epoch-aligned: a point at timestamp t (nanoseconds) is rewritten to floor(t / window) * window. Values and tag groupings are preserved; aggregation of points that collide on the snapped timestamp is the responsibility of downstream nodes.
inputEnum1dArraySeriesrequired
windowcompute.api.Durationrequired
EnumAggregation#
Aggregates an input series using an enum aggregation operator. Rolling aggregation builders are not supported; only the group-by shape is accepted at resolution time. Min and Max require a categorical input series. Udf also accepts a numeric input, which reaches the function as Vec<f64>; a categorical input reaches it as Vec<String>.
inputAggregationBuilderrequiredThe aggregation shape to apply the operator over. Only the group-by shape is supported; rolling builders are rejected at resolution time. The Python
enum_aggregation(...)sugar form is categorical-only. Numeric builders expose the UDF-onlyenum_udf(...)andrust_enum_udf(...)terminals instead.operatorEnumAggregationOperatorrequiredThe enum reduction applied to each group (min, max, or a user-defined function).
EnumAggregationFunction#
Aggregates duplicate timestamps on a single enum series. For aggregating across tag groupings, use EnumSeries.enumAggregation with aggregation.AggregationBuilder.groupBy.
MIN
MAX
EnumAggregationOperator#
Set type to the name of one of the other fields, and set only that field.
minEmpty
maxEmpty
udfEnumAggregationUdf
EnumAggregationUdf#
A user-defined enum aggregation. The supported ABI is a Rust function with one vector input and a String return value. Numeric input series use Vec<f64> and enum input series use Vec<String>.
sourceUdfSourcerequired
EnumConstantResampleInterpolationConfiguration#
Fills all empty resample intervals with a single constant value.
If a resampled time interval (bucket) contains no data points from the input, it will be assigned the specified ‘constant’ as a default value. This applies to gaps at the start of the series, between data points, and at the end.
constantStringConstantrequired
EnumCountDuplicateSeries#
Counts the number of points along each timestamp in the input series.
inputsarray ofEnumSeries
EnumFilterOperator#
IS_INTrue when the value is contained in the set.
IS_NOT_INTrue when the value is not contained in the set.
MATCHES_ANY_REGEXTrue when the value matches at least one pattern in the set. Values are RE2 regular expressions, anchored to the entire string.
EnumFilterRanges#
Produces a list of ranges for which the filter condition is satisfied.
inputEnumSeriesrequired
operatorEnumFilterOperatorrequired
valuesStringSetConstantrequiredThe set of literal values (for IS_IN / IS_NOT_IN) or RE2 regex patterns (for MATCHES_ANY_REGEX) to match against.
persistenceWindowConfigurationPersistenceWindowConfiguration
EnumFilterTransformationSeries#
Outputs categorical input values that fall within the supplied ranges.
inputEnumSeriesrequired
filterRangeSeriesrequiredKeeps only input points whose timestamp falls inside one of these ranges.
EnumNthPointDownsampleSeries#
Keeps every Nth row of the timestamp-ordered input, per series. No grid arithmetic, no interpolation, no boundary handling. Output count is ceil(input_count / n) per series. Caller is responsible for choosing n consistent with the input’s sample rate; output spacing is only uniform when the input is.
inputEnumSeriesrequired
nIntegerConstantrequiredKeep every Nth point starting from the earliest in each series. Must be >= 1.
EnumResampleConfiguration#
intervalDurationConstantrequiredInterval between resampled points
interpolationEnumResampleInterpolationConfigurationInterpolation strategy to use (defaults to forward fill).
EnumResampleInterpolationConfiguration#
Set type to the name of one of the other fields, and set only that field.
forwardFillResampleInterpolationConfigurationForwardFillResampleInterpolationConfiguration
constantResampleInterpolationConfigurationEnumConstantResampleInterpolationConfiguration
EnumResampleSeries#
Resamples the input series to a new resolution using interpolation. Outputs data for timestamps corresponding to the defined frequency. Based on interpolation strategy, determines range of timestamps to output data for and interpolates values where necessary.
inputEnumSeriesrequired
resampleConfigurationEnumResampleConfigurationrequiredThe interval at which to resample the series and interpolation strategy
EnumResampleSeriesV2#
Resamples categorical points at the authored interval using forward-fill interpolation.
inputEnumSeriesrequired
intervalcompute.api.DurationrequiredInterval at which to resample.
EnumSeries#
Set type to the name of one of the other fields, and set only that field.
aggregateAggregateEnumSeriesdeprecatedDeprecated: AggregateEnumSeries is deprecated. Use
enumAggregationwithaggregation.GroupByAggregationBuilderinstead.rawReference
channelChannelSeries
derivedDerivedSeries
resampleEnumResampleSeries
resampleV2EnumResampleSeriesV2
nthPointDownsampleEnumNthPointDownsampleSeries
timeRangeFilterEnumTimeRangeFilterSeries
timeShiftEnumTimeShiftSeries
timeShiftV2EnumTimeShiftSeriesV2
unionEnumUnionSeries
filterTransformationEnumFilterTransformationSeries
valueMapValueMapSeries
select1dArrayIndexSelectIndexFrom1dEnumArraySeries
extractFromStructExtractEnumFromStructSeries
eventAggregationEventsEnumSeries
enumAggregationEnumAggregation
selectEnumSelectSeries
selectCategoricalPropertySelectProperty
filterByTagEnumTagFilterSeries
selectTagsEnumSelectTagsSeries
scalarUdfScalarUdfSeries
toStartOfIntervalEnumToStartOfIntervalSeries
EnumSeriesEqualityRanges#
Produces a list of ranges for which provided enum series are all equal (or are not all equal).
inputarray ofEnumSeries
equalityOperatorEqualityOperatorrequired
persistenceWindowConfigurationPersistenceWindowConfiguration
interpolationConfigurationInterpolationConfigurationdeprecatedDefaults to forward fill interpolation with a 1s interpolation radius Deprecated: Setting interpolation on this node is deprecated.
EnumTagFilterSeries#
Applies a tag predicate to the enum series produced by input.
inputEnumSeriesrequired
predicateTagPredicaterequired
EnumTimeRangeFilterSeries#
Filters the series to points within the specified time range.
inputEnumSeriesrequired
startTimeTimestampConstantRepresents the start time (inclusive) of the time range.
endTimeTimestampConstantRepresents the end time (inclusive) of the time range.
EnumTimeShiftSeries#
inputEnumSeriesrequired
durationDurationConstantrequired
EnumTimeShiftSeriesV2#
Shifts the timestamps in the input series by the resolved duration.
inputEnumSeriesrequired
durationcompute.api.DurationrequiredDuration to shift the input series by.
EnumToNumericSeries#
Maps an enumerated series to a numeric series by mapping each string value to a double.
inputEnumSeriesrequired
mappingmap of string toDoubleConstantThe mapping from enum values to doubles.
defaultValueDoubleConstantThe value to use for enum values not present in the mapping. If not specified, points with unmapped enum values will be dropped.
EnumToStartOfIntervalSeries#
Snaps each point’s timestamp to the start of its enclosing fixed-size time bucket. Buckets are epoch-aligned: a point at timestamp t (nanoseconds) is rewritten to floor(t / window) * window. Values and tag groupings are preserved; aggregation of points that collide on the snapped timestamp is the responsibility of downstream nodes.
inputEnumSeriesrequired
windowcompute.api.Durationrequired
EnumUnionOperation#
THROWThrows an exception if two series contain points with a duplicate timestamp.
MAXAggregates duplicates by String MAX.
MINAggregates duplicates by String MIN.
EnumUnionSeries#
Combines multiple enum series together and outputs a single series. The strategy to merge input values with the same timestamp together is specified in the operation field.
inputarray ofEnumSeries
operationEnumUnionOperationrequiredThe strategy to merge points with duplicate timestamps.
EpochTimestamp#
timeUnitTimeUnitrequired
EqualityOperator#
ALL_EQUAL
NOT_ALL_EQUAL
EqualToSeries#
Compares two numeric series point-wise, producing true where left == right and false otherwise.
leftNumericSeriesrequired
rightNumericSeriesrequired
ErrorResult#
errorTypestring required
messagestring required
errorInstanceIdstring
EventAnchor#
Uses the start timestamp of the event at the configured zero-based index from this query whose event time lies in the anchor window [start, end), or [start, +infinity) when the end is absent. The window is the run bounds for a run anchor. Shares the wire shape of numeric.EventTimeShift, which tagByIntervals time shifts use; a separate type so dataset anchors evolve independently.
queryComputeEventQueryrequired
indexIntegerConstantZero-based event index after sorting. Defaults to zero and must be non-negative.
sortOrderEventSortOrderOrdering by event start timestamp before selecting the index. Defaults to ASC.
EventDispositionStatus#
PENDING_REVIEW
CLOSED_IGNORED
CLOSED_REQUIRES_FURTHER_ACTION
NO_DISPOSITION
EventIntervalSource#
Intervals come from an event query. Each event defines [timestamp, timestamp+duration).
queryComputeEventQueryrequired
tagByarray ofEventTagSourceNon-empty ordered list of tag sources. Each contributes a tag key-value pair per event interval.
EventLabelTag#
labelStringConstantrequired
EventLevelTag#
An empty JSON object, {}.
EventPropertyTag#
propertyNameStringConstantrequired
EventsEnumSeries#
Queries events and emits an enum series where each data point corresponds to an event and the enum value is derived from a single event field (property, label, etc).
queryComputeEventQueryrequired
valueSourceEventsEnumValueSourcerequired
EventsEnumValueSource#
Set type to the name of one of the other fields, and set only that field.
propertyStringConstantName of the event property whose value should be used as the enum value.
levelEmptyThe event level (INFO, FLAG, VIOLATION, SUCCESS). This field corresponds to the event type.
EventSortOrder#
ASC
DESC
EventsSearchRanges#
Produces a range series with a range for each event matching the query. Throws if there are more than 1,000 results.
queryComputeEventQueryrequired
EventTimeFilter#
timestampTimestampConstantrequired
timestampConditionEventTimeFilterConditionrequired
EventTimeFilterCondition#
START_TIME_BEFORE_INCLUSIVE
START_TIME_BEFORE_EXCLUSIVE
START_TIME_AFTER_INCLUSIVE
START_TIME_AFTER_EXCLUSIVE
END_TIME_BEFORE_INCLUSIVE
END_TIME_BEFORE_EXCLUSIVE
END_TIME_AFTER_INCLUSIVE
END_TIME_AFTER_EXCLUSIVE
EventTimeShift#
Shift by the start timestamp of the event at the configured zero-based index from this query whose event time lies in the anchor window [start, end), or [start, +infinity) when the end is absent. The window is the tagging interval for tagByIntervals and the run bounds for a run anchor. The query may differ from the query that produced a tagging interval.
queryComputeEventQueryrequired
indexIntegerConstantZero-based event index after sorting. Defaults to zero and must be non-negative.
sortOrderEventSortOrderOrdering by event start timestamp before selecting the index. Defaults to ASC.
EventType#
INFO
FLAG
ERROR
SUCCESS
EveryNthPerSeriesSampling#
Returns every Nth timestamp-ordered point from each distinct series.
nIntegerConstantrequiredKeep every Nth point starting from the earliest in each series. Must be > 1.
ExcludeNegativeValues#
An empty JSON object, {}.
ExponentialCurve#
An empty JSON object, {}.
ExternalConnectionMetadata#
connectionDetailsConnectionDetailsrequired
connectionStatusConnectionStatusrequired
limitsLimitsConfig
ExtractEnum1dArrayFromStructSeries#
inputStructSeriesrequired
fieldPatharray ofStringConstant
ExtractEnumFromStructSeries#
inputStructSeriesrequired
fieldPatharray ofStringConstantPath segments that are joined to locate a nested enum field. Use dots to traverse objects (e.g.
a.b.c) and 0-based brackets to index into arrays (e.g.items[0].value). A bare numeric name like0refers to an object key named “0”; use[0]to index into an array. Rows where the path is invalid or resolves to a nested object rather than a scalar are filtered out.
ExtractNumeric1dArrayFromStructSeries#
inputStructSeriesrequired
fieldPatharray ofStringConstant
ExtractNumericFromStructSeries#
inputStructSeriesrequired
fieldPatharray ofStringConstantPath segments that are joined to locate a nested numeric field. Use dots to traverse objects (e.g.
a.b.c) and 0-based brackets to index into arrays (e.g.items[0].value). A bare numeric name like0refers to an object key named “0”; use [0] to index into an array. Rows where the path is invalid or the value cannot be converted to the specified type are filtered out.dataTypeNumericDataTypeThe type the numeric data will be cast to, defaults to FLOAT64
ExtractStructFromStructSeries#
inputStructSeriesrequired
fieldPatharray ofStringConstantPath segments that are joined to locate a nested struct field. Use dots to traverse objects (e.g.
a.b.c) and 0-based brackets to index into arrays (e.g.items[0].value). A bare numeric name like0refers to an object key named “0”; use [0] to index into an array. Returns an empty struct when the path does not resolve to a valid JSON object.
ExtremaRangesV2#
Produces ranges for the selected extrema type.
inputNumericSeriesrequired
minimumProminenceDoubleConstantThe minimum topographic prominence for an extrema to be returned.
selectionPeakTyperequired
FillLimit#
How far forward in time a sample’s value may be carried.
Set type to the name of one of the other fields, and set only that field.
durationDurationConstant
FillStrategy#
Default strategy for filling gaps when aligning multi-input series.
Set type to the name of one of the other fields, and set only that field.
forwardFillForwardFill
FilterByExpressionSeries#
Keeps points from the base series where the boolean expression evaluates to true. Only outputs timestamps where all input series have an entry for that timestamp, or a value can be filled using the interpolation configuration.
basestring requiredThe variable name for the series that will be returned by this filter. The expression can filter points conditional on multiple series. The base defines which series’ points will be returned after applying all the filters.
inputsmap ofLocalVariableNametoNumericSeriesA map containing all the numeric series present in the expression.
expressionstring requiredAn expression that evaluates to a boolean. For example - a > 5.
interpolationConfigurationInterpolationConfigurationdeprecatedDefaults to forward fill interpolation with a 1s interpolation radius Deprecated: Setting interpolation on this node is deprecated.
FilteredDataset#
Applies a conjunctive tag predicate to the series produced by the inner dataset. Resolved by threading tag filters into channel specs and folding against known tag assignments; does not add a resolved numeric node.
inputapi.DatasetrequiredThe underlying dataset to filter.
predicateTagPredicaterequiredConjunctive tag predicate used to keep matching series.
FirstPointMatchingCondition#
The first point in the output range will be the first point where the condition evaluates to true.
An empty JSON object, {}.
FloorDivide#
Divides left by right pointwise and rounds down to the nearest integer.
leftNumericSeriesrequiredNumerator.
rightNumericSeriesrequiredDenominator.
ForwardFill#
At each grid point, carries the most recent prior sample value forward. The value is held only up to the configured limit; grid points with no prior sample within that window are left empty.
limitFillLimitrequired
ForwardFillInterpolation#
interpolationRadiusDurationConstantrequired
ForwardFillResampleInterpolationConfiguration#
Forward fill interpolation for resampling. Rounds the earliest and latest timestamp up to the nearest interval multiple and generates timestamps at the given interval between the new earliest and latest timestamps. For every timestamp in the resampled timestamps, takes the last known value before the timestamp as the value.
An empty JSON object, {}.
FunctionDerivedSeries#
A reference to a derived series resulting from applying a function to a set of arguments.
moduleNameStringConstantrequired
functionNameStringConstantrequired
versionReferenceModuleVersionReferencerequired
functionArgsmap ofFunctionParameterNametoFunctionParameterValueMap of function input names to their values. The function inputs must match the function’s parameter names and types. An input must be specified for each of the referenced function’s parameters.
FunctionParameterValue#
Set type to the name of one of the other fields, and set only that field.
variablestring
valueVariableValue
FunctionVariables#
variablesmap ofVariableNametoVariableValue
subFunctionVariablesmap ofFunctionReferencetoFunctionVariables
Granularity#
NANOSECONDS
GreaterThanOrEqualToSeries#
Compares two numeric series point-wise, producing true where left >= right and false otherwise.
leftNumericSeriesrequired
rightNumericSeriesrequired
GreaterThanSeries#
Compares two numeric series point-wise, producing true where left > right and false otherwise.
leftNumericSeriesrequired
rightNumericSeriesrequired
GroupByAggregationBuilder#
Groups points by a tag selection and a timestamp policy and applies the operator.
inputSeriesrequiredThe series to group and aggregate.
tagsGroupByTagsrequiredSelects which tag keys form the grouping key:
GroupByTags.Current()keeps the input series’ currently selected tags, orGroupByTags.Subset([...])restricts the grouping to a subset of them.timestampGroupByTimestamprequiredSelects how timestamps participate in the grouping key:
GroupByTimestamp.Current()groups on existing timestamps, andGroupByTimestamp.Every(duration)buckets points into fixed-size, epoch-aligned tumbling windows.alignmentGroupByAlignmentOptionally aligns source tag groups that map to the same output tag group before aggregation. Alignment is currently supported only with
GroupByTimestamp.Current()and built-in numeric aggregation operators.
GroupByAlignment#
Selects how source tag groups are aligned before aggregation.
Set type to the name of one of the other fields, and set only that field.
contextDefaultsEmptyUse the compute context’s
defaultAlignmentStrategyanddefaultFillStrategy.
GroupByTimestamp#
Selects how timestamps participate in the grouping key.
Set type to the name of one of the other fields, and set only that field.
currentEmptyUse the current timestamps.
everycompute.api.DurationBucket timestamps into a tumbling fixed-size, epoch-aligned window. A point at timestamp
t(nanoseconds) falls into the bucket[floor(t / window) * window, +window), and one aggregated point is emitted per bucket per tag grouping at the bucket start timestamp.
GroupWithinRangesSeriesV2#
Aggregates the input series within each supplied range and emits one point at the range start. Empty ranges emit no point, and ranges without a start use the compute request start.
inputNumericSeriesrequired
rangesRangeSeriesrequiredOne output point is emitted per range, at the range’s start.
aggregationNumericAggregationOperatorrequired
Handle#
Set type to the name of one of the other fields, and set only that field.
s3S3Handle
HashSubsampleSampling#
Keeps approximately 1/N of raw points via a stateless hash of each point’s timestamp.
nIntegerConstantrequiredKeep approximately one out of every N points. Must be > 1.
HeaderValue#
Set type to the name of one of the other fields, and set only that field.
plainstring
secretRidstring (RID)
HighPassConfiguration#
cutoffFrequencyDoubleConstantrequired
IdentityDimension#
Injects the value of a type’s primary key, under a tag key equal to the primary key’s property name by default. The primary key is read when the query runs, so it follows changes to the type. Only asset branches of an asset with the type have a value; every other branch, including run branches, gets the reserved value NOMINAL_ABSENT.
typeRidStringConstantrequiredResource identifier of the type whose primary key identifies the asset.
Influx1ConnectionDetails#
hoststring required
portinteger required
headersmap of string toHeaderValueA map of header name to value
credentialsPasswordCredentials
Influx2ConnectionDetails#
hoststring required
portinteger required
headersmap of string toHeaderValueA map of header name to value
orgstring required
tokenSecretRidstring (RID) requiredSecret Rid of token secret stored in Secrets Service.
InfluxChannelNameComponent#
Set type to the name of one of the other fields, and set only that field.
InfluxScrapingConfig#
filterarray ofInfluxScrapingFilterFilters are ANDed together
seriesArchetypeConstructionInfluxSeriesArchetypeConstructionConfigrequired
InfluxScrapingFilter#
Set type to the name of one of the other fields, and set only that field.
notInfluxScrapingFilter
matchBucketarray of stringMatch any of the specified buckets
matchMeasurementarray of stringMatch any of the specified measurements
InfluxSeriesArchetypeConstructionConfig#
For example, for config of the form {components: [measurement, field, tag: vehicle], separator: “.”} will construct a channel that looks like vehicle_a.measurement.field. Tags will automatically be specified inside of the archetype.
channelNameComponentsarray ofInfluxChannelNameComponent
separatorstring required
Ingesting#
progressnumberOptional fraction between 0 and 1 indicating progress through ingestion.
IngestStatus#
IN_PROGRESS
COMPLETED
FAILED
DELETION_IN_PROGRESS
DELETED
IngestStatusV2#
Set type to the name of one of the other fields, and set only that field.
successSuccessResult
errorErrorResult
inProgressInProgressResult
deletionInProgressDeletionInProgress
deletedDeleted
queuedQueued
parsingParsing
ingestingIngesting
IngestTagMetadata#
The tags used when ingesting the dataset file.
tagColumnsmap ofTagNameto stringA map of tag names to column names to derive the tag values from.
additionalFileTagsmap ofTagNameto string
InProgressDetails#
Sub-phase of an in-progress ingest. Absent on InProgressResult for a generic in-progress file; when present it indicates where in the pipeline the in-progress file currently is.
Set type to the name of one of the other fields, and set only that field.
InProgressResult#
inProgressDetailsInProgressDetails
IntegerConstant#
An integer constant that can be a literal value or a named variable reference.
Set type to the name of one of the other fields, and set only that field.
literalintegerCreate from a literal integer value.
variablestringCreate from a named reference name.
IntegralSeries#
Calculates the running sum of the area underneath a series using the trapezoidal rule.
inputNumericSeriesrequired
startTimestampTimestampConstantrequired
timeUnitTimeUnitTime unit used to calculate the integral. Defaults to seconds if not specified.
InterpolationConfiguration#
Set type to the name of one of the other fields, and set only that field.
forwardFillInterpolationForwardFillInterpolation
IntersectRanges#
Produces the intersection of the input ranges: the ranges of time covered by every input. For best performance, pass inputs in the order in which they should be evaluated, listing the ranges that are cheapest to compute first.
inputsarray ofRangeSeries
IntervalAnchor#
Selects which boundary of an interval to use as its anchor.
Set type to the name of one of the other fields, and set only that field.
intervalStartIntervalStartTimeShiftUses the interval’s start timestamp.
intervalEndIntervalEndTimeShiftUses the interval’s end timestamp.
eventInIntervalEventAnchor
rangeInIntervalRangesAnchor
IntervalEndTimeShift#
Shift by each tagging interval’s resolved end; resolution throws if end is missing.
An empty JSON object, {}.
IntervalIndexTag#
An empty JSON object, {}.
IntervalSource#
Where the intervals come from and how each is labeled.
Set type to the name of one of the other fields, and set only that field.
eventsEventIntervalSource
rangesRangeIntervalSource
IntervalStartTimeShift#
Shift by each tagging interval’s resolved start; resolution throws if start is missing.
An empty JSON object, {}.
IntervalStartTimestampTag#
An empty JSON object, {}.
Iso8601Timestamp#
An empty JSON object, {}.
LabelsFilter#
operatorSetOperatorrequired
labelsarray of string
LessThanOrEqualToSeries#
Compares two numeric series point-wise, producing true where left <= right and false otherwise.
leftNumericSeriesrequired
rightNumericSeriesrequired
LessThanSeries#
Compares two numeric series point-wise, producing true where left < right and false otherwise.
leftNumericSeriesrequired
rightNumericSeriesrequired
LimitsConfig#
maxQueryRangeconnection.api.Duration
LiteralRange#
A literal range of values.
startTimestampTimestampConstant
endTimestampTimestampConstant
tagsmap of string to string
LiteralRanges#
A list of literal ranges.
literalRangesarray ofLiteralRange
Ln#
Computes the natural logarithm (base e) of each value.
xNumericSeriesrequiredInput numeric series (positive values).
Log10#
Computes the base-10 logarithm of each value.
xNumericSeriesrequiredInput numeric series (positive values).
LogArgFilterOperator#
Matches against a single log arg.
keyStringConstantrequired
operatorLogRegexFilterOperatorrequired
LogarithmicCurve#
An empty JSON object, {}.
LogExactMatchCaseInsensitiveFilter#
Filters points such that the log message in each point contains an exact case-insensitive match of the provided token.
tokenstring required
LogFilterOperator#
Set type to the name of one of the other fields, and set only that field.
regexFilterLogRegexFilterOperator
exactMatchCaseInsensitiveFilterLogExactMatchCaseInsensitiveFilter
argLogArgFilterOperator
andarray ofLogFilterOperator
orarray ofLogFilterOperator
notLogFilterOperator
LogFilterSeries#
Outputs only values of the log plot that satisfy the filter.
inputLogSeriesrequired
operatorLogFilterOperatorrequired
LogRegexFilterOperator#
Matches a text value against the given re2 regular expression. Regular expression syntax: https://github.com/google/re2/wiki/Syntax.
regexstring required
LogSeries#
Set type to the name of one of the other fields, and set only that field.
rawReference
channelChannelSeries
derivedDerivedSeries
selectLogsSelectSeries
unionLogUnionSeries
filterLogFilterSeries
timeShiftLogTimeShiftSeries
toStartOfIntervalLogToStartOfIntervalSeries
filterByTagLogTagFilterSeries
selectTagsLogSelectTagsSeries
LogTagFilterSeries#
Applies a tag predicate to the log series produced by input.
inputLogSeriesrequired
predicateTagPredicaterequired
LogTimeShiftSeries#
inputLogSeriesrequired
durationDurationConstantrequired
LogToStartOfIntervalSeries#
Snaps each point’s timestamp to the start of its enclosing fixed-size time bucket. Buckets are epoch-aligned: a point at timestamp t (nanoseconds) is rewritten to floor(t / window) * window. Values and tag groupings are preserved; aggregation of points that collide on the snapped timestamp is the responsibility of downstream nodes.
inputLogSeriesrequired
windowcompute.api.Durationrequired
LogUnionOperation#
KEEP_ALLKeeps all points if multiple series contain points with a duplicate timestamp.
LogUnionSeries#
Combines multiple log series together and outputs a single series. The strategy to merge input values with the same timestamp together is specified in the operation field.
inputarray ofLogSeries
operationLogUnionOperationrequiredThe strategy to merge points with duplicate timestamps.
LowPassConfiguration#
cutoffFrequencyDoubleConstantrequired
MathConstant#
Math constants referenceable by name.
PIThe circle constant π (dimensionless).
TAUThe circle constant τ = 2π (dimensionless).
EEuler’s number e (dimensionless).
Maximum#
The maximum value of points inside the time window.
An empty JSON object, {}.
MaxSeries#
For every timestamp specified in the input series, outputs a value that is the maximum for that timestamp across all input series. Only outputs timestamps where all input series have an entry for that timestamp, subject to interpolation.
inputsarray ofNumericSeriesInput series across which to take the pointwise maximum
interpolationConfigurationInterpolationConfigurationdeprecatedDefaults to forward fill interpolation with a 1s interpolation radius Deprecated: Setting interpolation on this node is deprecated.
McapChannelLocator#
Locator for a channel in an mcap file. Channel name is not guaranteed to be unique, so channel ID should be used for mcap files with duplicate channel names.
Set type to the name of one of the other fields, and set only that field.
topicstring
idinteger
McapTimestampManifest#
Timestamps are derived from the mcap file containing the video frames.
mcapChannelLocatorMcapChannelLocatorrequired
MeanSeries#
For every timestamp specified in the input series, outputs a value that is the mean for that timestamp across all input series. Only outputs timestamps where all input series have an entry for that timestamp, subject to interpolation.
inputsarray ofNumericSeriesInput series across which to take the pointwise mean
interpolationConfigurationInterpolationConfigurationdeprecatedDefaults to forward fill interpolation with a 1s interpolation radius Deprecated: Setting interpolation on this node is deprecated.
Minimum#
The minimum value of points inside the time window.
An empty JSON object, {}.
MinMaxThresholdOperator#
OUTSIDE_BOUNDS
OUTSIDE_OR_EQUAL_TO_BOUNDS
INSIDE_BOUNDS
INSIDE_OR_EQUAL_TO_BOUNDS
MinMaxThresholdRanges#
Produces a list of ranges for which the threshold condition is satisfied.
inputNumericSeriesrequired
lowerBoundDoubleConstantrequired
upperBoundDoubleConstantrequired
operatorMinMaxThresholdOperatorrequired
persistenceWindowConfigurationPersistenceWindowConfiguration
MinSeries#
For every timestamp specified in the input series, outputs a value that is the minimum for that timestamp across all input series. Only outputs timestamps where all input series have an entry for that timestamp, subject to interpolation.
inputsarray ofNumericSeriesInput series across which to take the pointwise minimum
interpolationConfigurationInterpolationConfigurationdeprecatedDefaults to forward fill interpolation with a 1s interpolation radius Deprecated: Setting interpolation on this node is deprecated.
ModuleVersionReference#
Set type to the name of one of the other fields, and set only that field.
pinnedPinnedModuleVersionReference
Modulo#
Computes the remainder of left divided by right pointwise.
leftNumericSeriesrequiredDividend.
rightNumericSeriesrequiredDivisor.
Multiply#
Multiplies two numeric series pointwise.
leftNumericSeriesrequiredLeft-hand operand.
rightNumericSeriesrequiredRight-hand operand.
NamedSeries#
nameStringConstantrequiredName of the new derived series.
definitionSeriesrequiredCompute expression defining the series values.
Negate#
Negates each value in the series pointwise (multiplies by -1).
xNumericSeriesrequiredInput numeric series.
NegativeValueConfiguration#
Set type to the name of one of the other fields, and set only that field.
allowNegativeValuesAllowNegativeValuesInclude negative output values
setNegativeValuesToZeroSetNegativeValuesToZeroSet negative output values to zero
excludeNegativeValuesExcludeNegativeValuesExclude points with negative output values
nominal.api.Property#
namestring required
valuestring required
NominalChannelNameComponent#
Set type to the name of one of the other fields, and set only that field.
channelEmpty
valueOfTagWithNamestringThe value of the specified TagName will be used. If the tag is not present, it will be omitted.
NominalConnectionDetails#
nominalDataSourceRidstring (RID) required
NominalScrapingConfig#
channelNameComponentsarray ofNominalChannelNameComponentchannelNameComponents will be combined, together with separator, to form a fully qualified channel name.
separatorstring required
NotEqualToSeries#
Compares two numeric series point-wise, producing true where left != right and false otherwise.
leftNumericSeriesrequired
rightNumericSeriesrequired
NoTimestampManifest#
these values will not be updated after ingest time, to allow for resetting. The updated values are stored implicitly through the segment timestamps.
startingTimestampTimestamprequiredSpecifies the original starting timestamp of the video.
scaleParameterScaleParameterA field that specifies that the frame rate of the video does not match the frame rate of the camera | i.e. a slowed down or sped up video. Can specify either the camera frame rate or the absolute end time.
NotRanges#
The not ranges node will invert the ranges, filling the negative space in time.
inputRangeSeriesrequired
Numeric1dArraySeries#
Set type to the name of one of the other fields, and set only that field.
channelChannelSeries
rawReference
derivedDerivedSeries
selectNumeric1dArraySelectSeries
extractFromStructExtractNumeric1dArrayFromStructSeries
toStartOfIntervalNumeric1dArrayToStartOfIntervalSeries
timeShiftNumeric1dArrayTimeShiftSeries
filterByTagNumeric1dArrayTagFilterSeries
selectTagsNumeric1dArraySelectTagsSeries
Numeric1dArrayTagFilterSeries#
Applies a tag predicate to the numeric 1d array series produced by input.
inputNumeric1dArraySeriesrequired
predicateTagPredicaterequired
Numeric1dArrayTimeShiftSeries#
inputNumeric1dArraySeriesrequired
durationDurationConstantrequired
Numeric1dArrayToStartOfIntervalSeries#
Snaps each point’s timestamp to the start of its enclosing fixed-size time bucket. Buckets are epoch-aligned: a point at timestamp t (nanoseconds) is rewritten to floor(t / window) * window. Values and tag groupings are preserved; aggregation of points that collide on the snapped timestamp is the responsibility of downstream nodes.
inputNumeric1dArraySeriesrequired
windowcompute.api.Durationrequired
NumericAggregation#
Aggregates an input series using the given operator. The shape of the aggregation (rolling time window, rolling point-count window, or group by) is selected by the aggregation builder passed as input. The operator must be compatible with that builder’s series, producing a numeric output: count reduces a series of any type, whereas SUM requires a numeric series.
inputAggregationBuilderrequiredThe aggregation shape to apply the operator over: a rolling time window, a rolling point-count window, or a group-by.
operatorNumericAggregationOperatorrequiredThe reduction applied to each window or group (e.g. sum, mean, percentile). Must be compatible with the input series, producing a numeric output: Count works on any input series, whereas reductions like Sum require a numeric series.
NumericAggregationFunction#
DEPRECATED. Use NumericAggregationOperator instead.
SUM
MEAN
MIN
MAX
COUNT
STANDARD_DEVIATION
ROOT_MEAN_SQUARE
NumericAggregationOperator#
Unified numeric aggregation used across rolling windows, plain aggregates, range aggregates, union merges, and per-bucket summarization.
Set type to the name of one of the other fields, and set only that field.
sumSummation
averageAverage
minMinimum
maxMaximum
countCount
standardDeviationStandardDeviation
rootMeanSquareRootMeanSquare
percentilePercentile
udfNumericAggregationUdf
NumericAggregationUdf#
A user-defined numeric aggregation. The aggregation builder controls grouping/windowing. Supported Rust entrypoints accept grouped value vectors, optionally with leading sample timestamps, and return either one numeric value or timestamped numeric point(s), e.g. fn rms(values: Vec<f64>) -> f64, fn energy(timestamp_ns: Vec<i64>, values: Vec<f64>) -> f64, fn first(timestamp_ns: Vec<i64>, values: Vec<f64>) -> (i64, f64), or fn passthrough(timestamp_ns: Vec<i64>, values: Vec<f64>) -> Vec<(i64, f64)>. The value vector’s element type follows the aggregated series: Vec<f64> for a numeric series, and Vec<String> for a categorical or struct series, where each struct row arrives as a JSON object. A UDF over a combined struct series may instead take one vector per named struct field it needs, e.g. fn mean_product(channel0: Vec<f64>, channel1: Vec<f64>) -> f64. Parameter names select the fields, so a subset of the struct’s fields in any order is accepted.
sourceUdfSourcerequired
NumericApproximateFilterSeries#
Outputs the values of the numeric plot value that are approximately equal to the threshold value.
inputNumericSeriesrequired
thresholdDoubleConstantrequired
toleranceDoubleConstantrequired
operatorApproximateThresholdOperatorrequired
NumericConstantResampleInterpolationConfiguration#
Fills all empty resample intervals with a single constant value.
If a resampled time interval (bucket) contains no data points from the input, it will be assigned the specified ‘constant’ as a default value. This applies to gaps at the start of the series, between data points, and at the end.
constantDoubleConstantrequired
NumericDataType#
Supported numeric data types for value casting.
INT64
FLOAT64
UINT64
NumericEqualityRangesV2#
Produces ranges where all numeric inputs are equal or are not all equal.
inputsarray ofNumericSeriesThe series compared against each other. At least two are required.
toleranceDoubleConstantMaximum spread between the inputs still counted as equal. Defaults to zero, i.e. exact equality.
minPointsIntegerConstantMinimum number of matching points. Defaults to 1.
minDurationcompute.api.DurationMinimum matching duration. Defaults to one nanosecond.
rangeStartRangeStartSelects whether the output begins at the first match or after persistence is satisfied.
equalityOperatorEqualityOperatorrequired
NumericFilterTransformationSeries#
Outputs the input points whose timestamps fall within the supplied ranges.
inputNumericSeriesrequired
filterRangeSeriesrequiredKeeps input points whose timestamps fall inside any of these ranges.
NumericFilterWhereSeries#
Outputs numeric input values where the provided Boolean expression evaluates to true.
inputNumericSeriesrequired
conditionBooleanSeriesrequiredCondition evaluated point-wise at each input timestamp. A point is only kept where the condition is true.
NumericIfElseSeries#
Selects between two numeric series point-wise based on a boolean condition series. For every aligned timestamp, emits the value from trueValue where condition is true, and the value from falseValue otherwise.
conditionBooleanSeriesrequiredBoolean series evaluated at each point to choose between the
trueValueandfalseValuebranches.trueValueNumericSeriesrequiredValue emitted at points where
conditionis true.falseValueNumericSeriesrequiredValue emitted at points where
conditionis false.
NumericLargestTriangleThreeBucketsSeries#
Applies Largest-Triangle-Three-Buckets to a numeric time series for visualization-oriented downsampling. The algorithm preserves the first and last points, partitions the remaining input points based on the requested output count, and selects points that preserve the visual shape of the series. Results are selected input timestamp/value points, not fixed-width bucket aggregate rows.
inputNumericSeriesrequired
maxPointsToReturnIntegerConstantrequiredMaximum number of selected points to return. Must be positive and no greater than 10,000.
NumericNanFilter#
Removes points where the value is NaN from the input series.
inputNumericSeriesrequiredInput numeric series.
NumericNthPointDownsampleSeries#
Keeps every Nth row of the timestamp-ordered input, per series. No grid arithmetic, no interpolation, no boundary handling. Output count is ceil(input_count / n) per series. Caller is responsible for choosing n consistent with the input’s sample rate; output spacing is only uniform when the input is.
inputNumericSeriesrequired
nIntegerConstantrequiredKeep every Nth point starting from the earliest in each series. Must be >= 1.
NumericPropertyPredicate#
Matches resources that have a numeric value for the named property and whose value satisfies the operator. NEQ excludes resources that lack the property or store a string value under the same name.
namestring required
valuenumber required
operatorPropertyComparisonOperatorrequired
NumericPropertyRangeOperator#
BETWEEN
NOT_BETWEEN
NumericPropertyRangePredicate#
Inclusive range on a numeric property. At least one bound is required. BETWEEN: min <= x <= max. An absent bound is unbounded on that side. NOT_BETWEEN: x < min OR x > max. An absent bound drops that side. Both operators require a numeric value for name. Missing properties and string values never match. Bounds are closed [min, max] to match the number chip (is between / is not between), not half-open.
namestring required
minnumber
maxnumber
operatorNumericPropertyRangeOperatorrequired
NumericResampleConfiguration#
intervalDurationConstantrequiredInterval between resampled points
interpolationNumericResampleInterpolationConfigurationInterpolation strategy to use (defaults to forward fill).
NumericResampleInterpolationConfiguration#
Set type to the name of one of the other fields, and set only that field.
forwardFillResampleInterpolationConfigurationForwardFillResampleInterpolationConfiguration
constantResampleInterpolationConfigurationNumericConstantResampleInterpolationConfiguration
NumericResampleSeries#
Resamples the input series to a new resolution using interpolation. Outputs data for timestamps corresponding to the defined frequency. Based on interpolation strategy, determines range of timestamps to output data for and interpolates values where necessary.
inputNumericSeriesrequired
resampleConfigurationNumericResampleConfigurationrequiredThe interpolation strategy and interval at which to resample the series
NumericResampleSeriesV2#
Resamples the input series at the authored interval using forward-fill.
inputNumericSeriesrequired
intervalcompute.api.DurationrequiredInterval at which to resample.
NumericSeries#
Set type to the name of one of the other fields, and set only that field.
absAbs
negateNegate
cosCos
sinSin
tanTan
acosAcos
asinAsin
lnLn
log10Log10
sqrtSqrt
addAdd
subtractSubtract
multiplyMultiply
divideDivide
floorDivideFloorDivide
powerPower
moduloModulo
atan2Atan2
maxMaxSeries
meanMeanSeries
minMinSeries
sumSumSeries
unionNumericUnionSeries
productProductSeries
constantConstantNumericSeries
selectNewestPointsSelectNewestPointsSeries
selectNumericSelectSeries
selectNumericPropertySelectProperty
selectOldestPointsSelectOldestPointsSeries
bitAndBitAnd
bitOrBitOr
bitXorBitXor
bitShiftRightBitShiftRight
bitShiftLeftBitShiftLeft
bitTestBitTest
ifElseNumericIfElseSeries
numericAggregationNumericAggregation
countDuplicateEnumCountDuplicateSeries
cumulativeSumCumulativeSumSeries
derivativeDerivativeSeries
integralIntegralSeries
zScoreZscoreSeries
rawReference
derivedDerivedSeries
resampleNumericResampleSeries
resampleV2NumericResampleSeriesV2
nthPointDownsampleNumericNthPointDownsampleSeries
largestTriangleThreeBucketsNumericLargestTriangleThreeBucketsSeries
signalFilterSignalFilterSeries
timeDifferenceTimeDifferenceSeries
absoluteTimestampAbsoluteTimestampSeries
timeRangeFilterNumericTimeRangeFilterSeries
timeShiftNumericTimeShiftSeries
timeShiftV2NumericTimeShiftSeriesV2
unitConversionUnitConversionSeries
unitOverrideUnitOverrideSeries
valueDifferenceValueDifferenceSeries
filterTransformationNumericFilterTransformationSeries
filterWhereNumericFilterWhereSeries
thresholdFilterNumericThresholdFilterSeries
approximateFilterNumericApproximateFilterSeries
dropNanNumericNanFilter
select1dArrayIndexSelectIndexFrom1dNumericArraySeries
aggregateUnderRangesAggregateUnderRangesSeries
rangeDurationRangeDurationSeries
groupWithinRangesV2GroupWithinRangesSeriesV2
filterByExpressionFilterByExpressionSeries
scalarUdfScalarUdfSeries
enumToNumericEnumToNumericSeries
containsContainsSeries
refpropRefpropSeries
refpropV2RefpropSeriesV2
extractFromStructExtractNumericFromStructSeries
tagByIntervalsTagByIntervalsSeries
filterByTagNumericTagFilterSeries
selectTagsNumericSelectTagsSeries
toStartOfIntervalNumericToStartOfIntervalSeries
phaseUnwrapPhaseUnwrapSeries
aggregateAggregateNumericSeriesdeprecatedDeprecated: AggregateNumericSeries is deprecated. Use
numericAggregationwithaggregation.GroupByAggregationBuilderinstead.rollingOperationRollingOperationSeriesdeprecatedDeprecated: RollingOperationSeries is deprecated. Use
numericAggregationwithaggregation.RollingTimeAggregationBuilderinstead.bitOperationBitOperationSeriesdeprecatedDeprecated: BitOperationSeries is deprecated in favor of individual nodes for each operation.
channelChannelSeriesdeprecatedDeprecated: Deprecated in favor of the SelectSeries node
offsetOffsetSeriesdeprecatedDeprecated: OffsetSeries is deprecated in favor of add with a constant.
scaleScaleSeriesdeprecatedDeprecated: ScaleSeries is deprecated in favor of multiply with a constant.
arithmeticArithmeticSeriesdeprecatedDeprecated: ArithmeticSeries is deprecated in favor of individual nodes for each operation.
unaryArithmeticUnaryArithmeticSeriesdeprecatedDeprecated: UnaryArithmeticSeries is deprecated in favor of individual nodes for each operation.
binaryArithmeticBinaryArithmeticSeriesdeprecatedDeprecated: BinaryArithmeticSeries is deprecated in favor of individual nodes for each operation.
NumericStaleRangesV2#
Produces ranges where a numeric input has no data for at least a duration.
inputNumericSeriesrequired
thresholdcompute.api.DurationrequiredMinimum gap without data that produces an output range.
NumericTagFilterSeries#
Applies a tag predicate to the numeric series produced by input.
inputNumericSeriesrequired
predicateTagPredicaterequired
NumericThresholdFilterSeries#
Outputs only the values of the numeric plot value that satisfy the threshold condition.
inputNumericSeriesrequired
thresholdDoubleConstantrequired
operatorThresholdOperatorrequired
NumericTimeRangeFilterSeries#
Filters the series to points within the specified time range.
inputNumericSeriesrequired
startTimeTimestampConstantRepresents the start time (inclusive) of the time range.
endTimeTimestampConstantRepresents the end time (inclusive) of the time range.
NumericTimeShiftSeries#
inputNumericSeriesrequired
durationDurationConstantrequired
NumericTimeShiftSeriesV2#
Shifts the timestamps in the input series by the resolved duration.
inputNumericSeriesrequired
durationcompute.api.DurationrequiredDuration to shift the input series by.
NumericToStartOfIntervalSeries#
Snaps each point’s timestamp to the start of its enclosing fixed-size time bucket. Buckets are epoch-aligned: a point at timestamp t (nanoseconds) is rewritten to floor(t / window) * window. Values and tag groupings are preserved; aggregation of points that collide on the snapped timestamp is the responsibility of downstream nodes.
inputNumericSeriesrequired
windowcompute.api.Durationrequired
NumericUnionOperation#
DEPRECATED: apply aggregations as a separate step over the unioned output.
MINCombines multiple points by taking the minimum value
MAXCombines multiple points by taking the maximum value
MEANCombines multiple points by taking the mean of the values
SUMCombines multiple points by taking the sum of the values
COUNTCombines multiple points by taking the count of the values
STANDARD_DEVIATIONCombines multiple points by taking the standard deviation of the values
ROOT_MEAN_SQUARECombines multiple points by taking the root mean square of the values
THROWThrows a DuplicateTimestamp error if two series contain points with a duplicate timestamp.
NumericUnionSeries#
Combines multiple numeric series together and outputs a single series. By default, points from the inputs are concatenated (SQL {@code UNION ALL}) and duplicate timestamps across inputs are preserved. To aggregate values that share a timestamp, do a separate aggregation step next.
inputarray ofNumericSeriesInput series to union
operationNumericUnionOperationdeprecatedThe strategy to merge points with duplicate timestamps. Deprecated: Apply aggregations as a separate step over the unioned output.
OffsetSeries#
For every timestamp specified in the input series, offset it by a constant factor.
inputNumericSeriesrequired
scalarDoubleConstantrequiredThe constant to add to each point
OnChangeRanges#
Produces a list of ranges for each point that has a different value to the previous point. A range will have identical start and end times.
inputSeriesrequired
OriginFileUri#
Pre-signed URI that can be used to download an origin file directly. Expires if the download has not been initiated within 1 minute.
uristring required
pathstring required
OrSeries#
Combines two boolean series point-wise using logical OR, producing true where either input is true and false otherwise.
leftBooleanSeriesrequired
rightBooleanSeriesrequired
OutputRangeStart#
Set type to the name of one of the other fields, and set only that field.
firstPointMatchingConditionFirstPointMatchingConditionThe first point in the output range will be the first point where the condition evaluates to true.
afterPersistenceWindowAfterPersistenceWindowThe output range will contain points where the condition has been true for at least the min duration and min points.
PaddedRanges#
Expands ranges durations by adding temporal padding to the start, end, or both sides. Note that if the input ranges contain a RangeValue, it will be dropped.
inputRangeSeriesrequired
paddingDurationConstantrequired
paddingConfigurationRangePaddingConfigurationConfiguration for how to apply padding to the ranges. Defaults to PAD_START_AND_END if not specified.
PaddedRangesV2#
Expands ranges using an authored Duration expression and a selected padding direction.
inputRangeSeriesrequired
durationcompute.api.DurationrequiredHow far to extend each range on every padded side.
paddingConfigurationRangePaddingConfigurationrequired
Parsing#
An empty JSON object, {}.
PasswordCredentials#
usernamestring required
passwordSecretRidstring (RID) requiredSecret Rid of password secret stored in Secrets Service.
PeakRanges#
Produces a list of ranges for each point that is greater than its neighbors. Peaks at edges are discarded, and continuous, multivalue, flat peaks will return all values.
inputNumericSeriesrequired
returnsPeaksboolean deprecatedTrue if returning peaks, else troughs. Deprecated: No longer used, use returnType instead
returnTypePeakTypeOptional for backcompatibility.
minimumProminenceDoubleConstantThe minimum topographic prominence for an extrema to be returned. Prominence is the minimum vertical distance needed to travel from an extrema to one of greater magnitude.
PeakType#
PEAK
TROUGH
BOTH
PercentageThreshold#
Threshold defined as the percentage of a given value.
valueDoubleConstantrequiredAllowed spread, as a percentage of the window’s mean value.
Percentile#
The value at the specified percentile.
percentileDoubleConstantrequiredPercentile to compute, in range [0, 1]. E.g. 0.5 for median, 0.95 for p95.
PersistenceWindowConfiguration#
Configures how long a condition has to be true for to output a time range, and what to use as the start of the output range. For a point to be included in the output time range, both the minPoints and minDuration conditions must be satisfied.
minPointsIntegerConstantThe minimum number of points for which this condition be must satisfied to include the time range in the output. Must be non-negative. If not present, will default to 1.
minDurationDurationConstantThe minimum number of points for which this condition must be satisfied to include the time range in the output. Must be non-negative. If not present, will default to 1 nanosecond.
outputRangeStartOutputRangeStartrequiredWhich point to use as the start of the output range. Defaults to firstPointMatchingCondition if not specified.
PhaseUnwrapSeries#
Removes discontinuities from a wrapping signal (e.g. phase or heading) by adding whole multiples of period whenever the step between consecutive samples exceeds half the period. Output length equals input length.
inputNumericSeriesrequired
periodDoubleConstantrequiredThe wrapping period of the signal (e.g. 2*pi for radians, 360 for degrees). A jump between consecutive samples greater than
period/2in magnitude is corrected by adding/subtracting a whole multiple ofperiod.
PhysicalConstant#
Physical constants referenceable by name. The server resolves the symbol to its numeric value in SI units.
SPEED_OF_LIGHTSpeed of light in vacuum, 299792458 m/s (exact).
AVOGADROAvogadro constant, 6.02214076e23 mol^-1 (exact).
BOLTZMANNBoltzmann constant, 1.380649e-23 J/K (exact).
PLANCKPlanck constant, 6.62607015e-34 J*s (exact).
ELEMENTARY_CHARGEElementary charge, 1.602176634e-19 C (exact).
GAS_CONSTANTMolar gas constant, 8.31446261815324 J/(mol*K) (exact, Avogadro times Boltzmann).
STANDARD_GRAVITYStandard acceleration of gravity, 9.80665 m/s^2 (exact by convention).
STANDARD_ATMOSPHEREStandard atmosphere, 101325 Pa (exact by convention).
ABSOLUTE_ZERO_CELSIUSAbsolute zero expressed in degrees Celsius, -273.15 (exact).
GRAVITATIONAL_CONSTANTNewtonian constant of gravitation, 6.67430e-11 m^3/(kg*s^2) (CODATA 2022, measured).
STEFAN_BOLTZMANNStefan-Boltzmann constant, 5.670374419e-8 W/(m^2*K^4) (exact derived).
PinnedModuleVersionReference#
versionStringConstantrequired
PivotedTimescaleChannelNameComponent#
Set type to the name of one of the other fields, and set only that field.
PivotedTimescaleScrapingConfig#
This config is used to scrape data from a Timescale database that has a pivoted schema. time | name | value | device 1 | temperature | 1 | a
filterarray ofTimescaleScrapingFilterIn order for data to be picked up by the scraper, it must match all filters in this list. To exclude chunks of data, it’s possible to use a not filter.
timeColumnstring requiredThe name of the column that holds the timestamp.
nameColumnstring requiredThe name of the column that holds the series name.
valueColumnstring requiredThe name of the column that holds the series values.
dimensionColumnsarray of stringThe names of the columns that comprise a dimension. They should have a database index for efficient filtering. We do not discover dimensions based on hypertable schema because they are not necessarily configured properly.
channelNameComponentsarray ofPivotedTimescaleChannelNameComponentchannelNameComponents will be combined, together with separator, to form a fully qualified channel name.
separatorstring required
PolynomialCurve#
degreeIntegerConstantrequiredThe highest allowable degree of the fit polynomial.
interceptDoubleConstantThe y-value at the point x (or t) = 0. If omitted, the y-intercept will also be fit to the data.
PolynomialCurveFit#
A polynomial fit of the enclosing target (y) against the chosen independent variable (x).
independentVariableCurveFitVariablerequiredThe
xaxis the target is fit against — either relative sample time or another series.degreeIntegerConstantrequiredThe highest allowable degree of the fit polynomial.
interceptDoubleConstantThe y-value at the point x (or t) = 0. If omitted, the y-intercept will also be fit to the data.
Power#
Raises base to the power of exponent pointwise.
baseNumericSeriesrequiredBase values.
exponentNumericSeriesrequiredExponent values.
PowerCurve#
An empty JSON object, {}.
ProductSeries#
For every timestamp specified in the input series, outputs a value that is the product for that timestamp across all input series. Only outputs timestamps where all input series have an entry for that timestamp, subject to interpolation.
inputsarray ofNumericSeriesInput series across which to take the pointwise product
interpolationConfigurationInterpolationConfigurationdeprecatedDefaults to forward fill interpolation with a 1s interpolation radius Deprecated: Setting interpolation on this node is deprecated.
PropertiesFilter#
namestring required
valuesarray of string
PropertyComparisonOperator#
EQ
NEQ
GT
GTE
LT
LTE
Queued#
An empty JSON object, {}.
RangeAggregationOperation#
Set type to the name of one of the other fields, and set only that field.
sumSummation
rootMeanSquareRootMeanSquare
averageAverage
minMinimum
maxMaximum
standardDeviationStandardDeviation
countCount
allEmpty
RangeDurationSeries#
Outputs a numeric series with one point per range: timestamp = range start, value = range duration.
inputRangeSeriesrequired
unboundedRangeBehaviorUnboundedRangeBehaviorrequiredHow to handle ranges with a missing start or end.
timeUnitTimeUnitThe time unit of the duration. Defaults to seconds if not specified.
RangeIntervalSource#
Intervals come from a range series. Must resolve to literal ranges.
rangesRangeSeriesrequired
tagByarray ofRangeTagSourceNon-empty ordered list of tag sources. Each contributes a tag key-value pair per range interval.
RangeMap#
startDoubleConstantInclusive start value. If not specified, the start is the prior range’s end value, or negative infinity.
endDoubleConstantExclusive end value. If not specified, the end value is the next range’s start value, or positive infinity.
outputStringConstantrequiredThe value to map to if the input value is within the range.
RangePaddingConfiguration#
PAD_START
PAD_END
PAD_START_AND_END
RangesAnchor#
Uses the start timestamp of the range at the configured zero-based index from ranges evaluated over each interval, considering only ranges whose start lies in the anchor window [start, end), or [start, +infinity) when the end is absent. The window is the run bounds for a run anchor. A range already in progress when the window opens anchors at the window start. Series in ranges that select no explicit dataset resolve against the aligned input dataset, and the computed range groups must carry the aligned runs’ runRid grouping tags.
rangesRangeSeriesrequiredThe range series evaluated per interval to locate the anchor range.
indexIntegerConstantZero-based range index after sorting. Defaults to zero and must be non-negative.
sortOrderRangeSortOrderOrdering by range start timestamp before selecting the index. Defaults to ASC.
RangeSeries#
Set type to the name of one of the other fields, and set only that field.
approximateThresholdApproximateThresholdRanges
booleanRangesBooleanRanges
thresholdRangesV2ThresholdRangesV2
durationFilterDurationFilterRanges
durationFilterV2DurationFilterRangesV2
enumFilterEnumFilterRanges
categoryFilterRangesV2CategoryFilterRangesV2
categoricalOnChangeRangesV2CategoricalOnChangeRangesV2
categoricalStaleRangesV2CategoricalStaleRangesV2
categoricalEqualityRangesV2CategoricalEqualityRangesV2
enumSeriesEqualityRangesNodeEnumSeriesEqualityRanges
eventsSearchEventsSearchRanges
intersectRangeIntersectRanges
literalRangesLiteralRanges
minMaxThresholdMinMaxThresholdRangesComputes ranges where the input time series matches a filter defined by lower and upper bounds, and an operator.
notNotRanges
onChangeOnChangeRanges
peakPeakRanges
extremaRangesV2ExtremaRangesV2
rangeNumericAggregationRangesNumericAggregation
rawReference
derivedDerivedSeries
seriesCrossoverRangesNodeSeriesCrossoverRanges
seriesEqualityRangesNodeSeriesEqualityRanges
numericEqualityRangesV2NumericEqualityRangesV2
stabilityDetectionStabilityDetectionRanges
stabilityDetectionV2StabilityDetectionRangesV2
staleRangeStaleRanges
numericStaleRangesV2NumericStaleRangesV2
thresholdThresholdingRangesComputes ranges where the input time series matches a filter defined by a single threshold and an operator.
unionRangeUnionRanges
paddedRangesPaddedRanges
paddedRangesV2PaddedRangesV2
RangesNumericAggregation#
Aggregates the values of a numeric series at each range specified by the input ranges.
rangesRangeSeriesrequired
inputNumericSeriesrequired
operationRangeAggregationOperationrequired
RangeSortOrder#
Sort order for selecting ranges. ASC selects from earliest to latest, DESC selects from latest to earliest.
ASC
DESC
RangeStart#
Selects which point of a satisfied condition begins the output range.
CONDITION_START
PERSISTENCE_START
Reference#
namestring requiredA reference to a value not defined in the current compute scope.
RefpropProperty#
Supported properties for REFPROP calculations.
TEMPERATURE
PRESSURE
MASS_DENSITY
MASS_SPECIFIC_ENTHALPY
MASS_SPECIFIC_INTERNAL_ENERGY
MASS_SPECIFIC_ENTROPY
VISCOSITY
RefpropSeries#
Computes the output property given input properties for the substance using REFPROP. Initial implementation only expects two input properties.
inputsmap of string toNumericSeriesA map that maps a REFPROP property to its numeric series.
outputPropertyRefpropPropertyrequiredThe desired output property. This should not be one of the input properties.
substanceRefpropSubstancerequiredThe substance for REFPROP calculations.
interpolationConfigurationInterpolationConfigurationdeprecatedDeprecated: Setting interpolation on this node is deprecated.
RefpropSeriesV2#
Computes a REFPROP output property from the supplied substance and output property.
inputsmap of string toNumericSeriesMap of REFPROP property to its numeric series.
substanceRefpropSubstancerequiredThe substance for REFPROP calculations.
outputPropertyRefpropPropertyrequiredThe desired output property. This must not be one of the input properties.
RefpropSubstance#
Supported substances for REFPROP calculations
NITROGEN
OXYGEN
HELIUM
HYDROGEN
METHANE
PROPANE
BUTANE
WATER
XENON
AIR
CARBON_DIOXIDE
ARGON
CARBON_MONOXIDE
JET_A_4658
JET_A_3638
RelativeTimestamp#
timeUnitTimeUnitrequired
offsetstring (date-time)
ResourceSearchQuery#
Predicate tree for filtering.
Set type to the name of one of the other fields, and set only that field.
propertiesPropertiesFilter
numericPropertyNumericPropertyPredicate
numericPropertyRangeNumericPropertyRangePredicateMatches resources using an inclusive numeric property range.
labelsLabelsFilter
workspaceStringConstant
andarray ofResourceSearchQuery
orarray ofResourceSearchQuery
notResourceSearchQuery
RetentionPolicy#
typeRetentionPolicyTyperequired
expiryDaysinteger
RetentionPolicyType#
KEEP_UNTIL_EXPIRY
KEEP_FOREVER
rids.api.Property#
nameStringConstantrequired
valueStringConstantrequired
RollingOperationSeries#
DEPRECATED. Use numericAggregation with aggregation.RollingTimeAggregationBuilder instead.
inputNumericSeriesrequired
windowWindowrequired
operatorNumericAggregationOperatorrequired
RollingPointsAggregationBuilder#
Rolling point-count aggregation: for each point in the input, aggregate the current point and the preceding N - 1 points within the same tag grouping.
inputSeriesrequired
pointsIntegerConstantrequired
RollingTimeAggregationBuilder#
Rolling time-window aggregation: for each point in the input, aggregate the values within the preceding time window.
inputSeriesrequired
windowcompute.api.Durationrequired
RootMeanSquare#
The root mean square of points inside the time window.
An empty JSON object, {}.
Run#
Run reference for Dataset.run. The rid field accepts either a literal run RID or a StringConstant reference resolved from Context.
ridStringConstantrequiredResource identifier of a run
run.api.Duration#
secondsinteger (int64) required
nanosinteger (int64) required
RunChannel#
runRidStringConstantrequired
assetRidStringConstantUsed to disambiguate when multiple assets within this run contain data scopes with the same name. If not specified for a run with multiple assets, an error will be thrown.
dataScopeNameStringConstantrequiredUsed to disambiguate when multiple data scopes within this run contain channels with the same name.
channelStringConstantrequired
additionalTagsmap of string toStringConstantdeprecatedDeprecated: Use additionalTagFilters
additionalTagFiltersTagFiltersTags to filter the channel by, in addition to tag filters defined for a given Run data scope. Throws on collisions with tag keys already defined for the given Run data scope. Only returns points that match both sets of tag filters. For log series, include arg filters here in addition to tag filters.
tagsToGroupByarray of string deprecatedDeprecated: Use groupByTags instead. Throws if both tagsToGroupBy and groupByTags are specified.
groupByTagsarray ofStringConstantTags that the channel should be grouped by. If this is non-empty a grouped result will be returned with an entry for each grouping. Only one of tagsToGroupBy and groupByTags should be specified.
RunsSearchTarget#
Search runs. Each matched run expands to one grouping per data source attached to the run: asset-typed data sources contribute one grouping per data scope on the asset; dataset-typed data sources contribute one grouping per dataset. Every grouping is tagged by runRid and dataScope (the asset’s data scope name for asset-typed sources, or the run-local data source ref name for dataset-typed sources); asset-typed groupings additionally carry assetRid.
queryResourceSearchQueryrequiredPredicate tree used to filter matching runs.
RustUdfSource#
Rust source code defining the UDF entrypoint and any helper functions.
sourceCodestring requiredRust source code to transpile into WASM. The operation consuming this source defines the entrypoint signature: scalar UDFs receive one scalar per input series, while aggregation UDFs receive grouped value vectors. If several functions appear at the top level, annotate the entrypoint with
#[main].
S3Handle#
bucketstring required
keystring required
Sampling#
Options for returning a sampled subset of raw series points.
Set type to the name of one of the other fields, and set only that field.
everyNthPerSeriesEveryNthPerSeriesSampling
hashSubsampleHashSubsampleSampling
SavedDataset#
A persisted source of data, named by its RID. Can be a catalog dataset, pipeline, connection, etc.
ridStringConstantrequiredA data source RID, given as a literal or as a variable resolved from
Context. A pipeline’s logic only accepts catalog datasets/pipelines, since it is persisted without context.
ScalarUdfSeries#
Executes user-provided scalar UDF code over aligned inputs. The function must return one output for each aligned input row, with the return type matching the series type that contains this node.
ScaleParameter#
Set type to the name of one of the other fields, and set only that field.
trueFrameRatenumber
endingTimestampTimestamp
scaleFactornumberthe scale factor can be used to calculate whether media duration differs from a video’s | real duration, and if so, the true frame rate of the camera. The video time will thus be scaled | by the ratio of the real duration to media duration, or media frame rate to true frame rate.
ScaleSeries#
For every timestamp specified in the input series, multiply it by a constant factor.
inputNumericSeriesrequired
scalarDoubleConstantrequiredThe constant to multiply each point by
scalarUnitstringThe units of the scalar to multiply by. If empty, the scalar is considered unit-less.
Scatter#
A scatter plot comprised of the values from one input series on the x-axis and values from another input series on the y-axis, for all points within a requested time range.
xNumericSeriesrequired
yNumericSeriesrequired
ScatterCurveFit#
Fits a curve to the scatter between two series.
scatterScatterrequiredThe x and y series to fit to.
optionsScatterFitOptionsrequired
ScatterFitOptions#
minXDoubleConstantInclusive lower bound on the x values to fit to. If omitted, does not set a bound (equivalent to setting this to the minimum x value among all points in the time range).
maxXDoubleConstantInclusive upper bound on the x values to fit to. If omitted, does not set a bound (equivalent to setting this to the maximum x value among all points in the time range).
minYDoubleConstantInclusive lower bound on the y values to fit to. If omitted, does not set a bound (equivalent to setting this to the minimum y value among all points in the time range).
maxYDoubleConstantInclusive upper bound on the y values to fit to. If omitted, does not set a bound (equivalent to setting this to the maximum y value among all points in the time range).
ScrapingConfig#
Set type to the name of one of the other fields, and set only that field.
influxInfluxScrapingConfig
nominalNominalScrapingConfig
timestreamTimestreamScrapingConfig
timescalePivotedTimescaleScrapingConfig
visualCrossingVisualCrossingScrapingConfig
bigQueryBigQueryScrapingConfig
apiApiScrapingConfig
SearchDataset#
Resolves matching assets or runs from search, creating a combined dataset tagged by source rid and data scope.
targetSearchTargetrequiredWhether to search assets or runs and with which query.
maxResultsintegerMaximum number of assets or runs returned from search. Defaults to 10000 when omitted.
SearchDatasetFilesQuery#
Set type to the name of one of the other fields, and set only that field.
timeRangeTimeRangeFilter
uploadedAtRangeUploadedAtFilter
fileTagsmap ofTagNameto string
andarray ofSearchDatasetFilesQuery
orarray ofSearchDatasetFilesQuery
SearchDatasetsQuery#
Set type to the name of one of the other fields, and set only that field.
searchTextstring
exactMatchstringPerforms case insensitive exact match search on the name.
labelstring
propertiesnominal.api.Property
numericPropertyNumericPropertyPredicateMatches datasets using a numeric property comparison.
numericPropertyRangeNumericPropertyRangePredicateMatches datasets using an inclusive numeric property range.
ingestStatusIngestStatus
ingestedBeforeInclusivestring (date-time)Inclusive upper bound on dataset creation time (ingestDate), not lastIngestedAt.
ingestedAfterInclusivestring (date-time)Inclusive lower bound on dataset creation time (ingestDate), not lastIngestedAt.
archiveStatusboolean
derivedbooleanFilters datasets by whether they have a derived definition.
andarray ofSearchDatasetsQuery
orarray ofSearchDatasetsQuery
workspacestring (RID)
SearchEventOriginType#
WORKBOOK
TEMPLATE
API
DATA_REVIEW
PROCEDURE
SearchTarget#
Set type to the name of one of the other fields, and set only that field.
assetsAssetsSearchTarget
runsRunsSearchTarget
SelectIndexFrom1dEnumArraySeries#
For each timestamp, selects a single enum value from the 1D enum array at the specified index. If the index is out of bounds for an array at a given timestamp, it is omitted.
inputEnum1dArraySeriesrequired
indexIntegerConstantrequired
SelectIndexFrom1dNumericArraySeries#
For each timestamp, selects a single numeric value from the 1D numeric array at the specified index. If the index is out of bounds for an array at a given timestamp, it is omitted.
inputNumeric1dArraySeriesrequired
indexIntegerConstantrequired
SelectNewestPointsSeries#
Select the most recent N points from the input series by timestamp.
inputNumericSeriesrequiredThe input series to select the newest points from.
numPointsIntegerConstantrequiredThe number of points to select.
SelectOldestPointsSeries#
Select the earliest N points from the input series by timestamp.
inputNumericSeriesrequiredThe input series to select the oldest points from.
numPointsIntegerConstantrequiredThe number of points to select.
SelectProperty#
Selects a typed property value from every asset or run branch resolved from dataset. Inside a WithSeriesDataset definition, dataset may be omitted to use that node’s wrapping input; it is required everywhere else. Asset branches read properties directly from their asset. Run branches read properties directly from their run, never from a backing asset or dataset. Direct saved-dataset branches are unsupported.
propertyNamestring required
datasetapi.Dataset
SelectSeries#
Selects series from a dataset (might contain multiple tag groupings). When name is provided, selects that single channel. When name is omitted, selection starts from all channels in the dataset scope; callers narrow the selection with a filterByTag predicate on the reserved NOMINAL_CHANNEL tag key and preserve per-channel identity in the response with selectTags. Providing both name and a NOMINAL_CHANNEL filter is rejected as an invalid request. Raw and derived channels can both be selected through the reserved tag filter.
nameStringConstantThe identifying series name to query
datasetapi.DatasetThe dataset providing the data to query. May be omitted inside a
WithSeriesDatasetseries definition, in which case it defaults to that node’s wrappinginput; required everywhere else.samplingSamplingRaw-series sampling options. Supported for numeric and enum raw series.
Series#
Set type to the name of one of the other fields, and set only that field.
rawReferencedeprecatedDeprecated: use typed value of series instead.
enumEnumSeries
numericNumericSeries
logLogSeries
arrayArraySeries
structStructSeries
videoReference
SeriesCrossoverRanges#
Produces a list of zero-duration ranges at the first point where two series cross over one another
inputANumericSeriesrequired
inputBNumericSeriesrequiredThe series
inputAis compared against.
SeriesEqualityRanges#
Produces a list of ranges for which provided series are all equal (or are not all equal).
inputarray ofNumericSeries
equalityOperatorEqualityOperatorrequired
toleranceDoubleConstant
persistenceWindowConfigurationPersistenceWindowConfiguration
interpolationConfigurationInterpolationConfigurationdeprecatedDefaults to forward fill interpolation with a 1s interpolation radius Deprecated: Setting interpolation on this node is deprecated.
SeriesVariable#
A series selected from a dataset, bound to a variable. The series type (numeric, enum, …) is decided lazily by the node consuming the variable, matching the behaviour of channel variables.
selectSelectSeriesrequired
groupByTagsarray ofStringConstantTags that the series should be grouped by. If this is non-empty a grouped result will be returned with an entry for each grouping; otherwise all matching series are merged.
SetNegativeValuesToZero#
An empty JSON object, {}.
SetOperator#
AND
OR
SignalFilterConfiguration#
Set type to the name of one of the other fields, and set only that field.
lowPassLowPassConfiguration
highPassHighPassConfiguration
bandPassBandPassConfiguration
bandStopBandStopConfiguration
SignalFilterSeries#
Applies IIR-based signal filtering to input series. Includes low-pass, high-pass, band-pass, and band-stop filters. Currently supports variable-order bidirectional Butterworth filters, with fixed-size padding based on SciPy output.
inputNumericSeriesrequired
signalFilterConfigurationSignalFilterConfigurationrequiredFilter type and cutoff frequencies.
orderIntegerConstantrequiredOrder of filter. Must be a positive integer, and is effectively doubled for bidirectional filters.
samplingFrequencyDoubleConstantThe sampling frequency of the input series. Used to calculate normalized frequency for cutoff frequencies.
Sin#
Computes the trigonometric sine of each value (input in radians).
xNumericSeriesrequiredInput numeric series (radians).
SortField#
INGEST_DATEDataset creation time (created_at). The name is historical; use LAST_INGESTED_AT for when data was last added.
START_TIMEStart of the dataset’s data bounds.
END_TIMEEnd of the dataset’s data bounds.
LAST_INGESTED_ATWhen data was last added, as the latest of a successful file ingest and streaming-session activity. Datasets that have never received either sort last.
SortOptions#
isDescendingboolean required
fieldSortFieldrequired
Sqrt#
Computes the square root of each value.
xNumericSeriesrequiredInput numeric series (non-negative values).
StabilityDetectionRanges#
Outputs a set of ranges where the input series is stable. For each point, the min and max are calculated over the specified lookback window, including the current point. A point is considered stable if its value does not deviate from the calculated min and the max by more than the threshold and the total number of points within the window is at least the specified amount. The threshold can be either fixed values or percentages of the value. The lookback window must be strictly positive. The minimum points threshold defaults to 2.
inputNumericSeriesrequired
windowConfigurationStabilityWindowConfigurationrequired
thresholdThresholdrequired
StabilityDetectionRangesV2#
Produces ranges where a numeric input is stable within a modern duration window.
inputNumericSeriesrequired
windowcompute.api.DurationrequiredLookback window over which the input’s spread is measured.
minPointsIntegerConstantMinimum number of points in the window. Defaults to 2.
thresholdThresholdrequired
StabilityWindowConfiguration#
minPointsIntegerConstantThe minimum number of points within the window to create a stable range. Must be non-negative. If not present, will default to 2.
durationDurationConstantrequired
StaleRanges#
Produces a list of ranges for which data does not exist for the specified duration or longer. Increases window size by the specified staleness threshold on both ends to capture edge cases of data not currently in view.
inputSeriesrequired
thresholdDurationConstantrequired
startTimestampTimestampConstantThe start timestamp of the range. If not specified, staleness will automatically use view range start.
endTimestampTimestampConstantdeprecatedDeprecated: No longer used and will be removed
StandardDeviation#
The standard deviation of points inside the time window.
An empty JSON object, {}.
StringConstant#
A string constant that can be a literal value or a named variable reference.
Set type to the name of one of the other fields, and set only that field.
literalstringCreate from a literal string value.
variablestringCreate from a named reference.
StringPropertyDimension#
Injects the value of a string property, under a tag key equal to the property name by default. Only supported on asset and run branches: asset branches read the asset’s properties, and run branches, including run branches backed by an asset, read the run’s properties, matching SelectProperty. Values are injected as is. A branch without the property, or whose property holds a numeric value, gets the reserved value NOMINAL_ABSENT.
propertyNameStringConstantrequiredName of the property whose value to inject. Also the default tag key.
StringSetConstant#
Set type to the name of one of the other fields, and set only that field.
literalarray of string
variablestring
StringSetConstantV2#
A set of string constants that can be a literal set or a named variable reference.
Set type to the name of one of the other fields, and set only that field.
literalarray ofStringConstantCreate from a literal set of string constants.
variablestringCreate from a named reference.
StructSeries#
Set type to the name of one of the other fields, and set only that field.
channelChannelSeries
rawReference
derivedDerivedSeries
combineCombineStructSeries
selectStructSelectSeries
extractFromStructExtractStructFromStructSeries
toStartOfIntervalStructToStartOfIntervalSeries
timeShiftStructTimeShiftSeries
filterByTagStructTagFilterSeries
selectTagsStructSelectTagsSeries
StructTagFilterSeries#
Applies a tag predicate to the struct series produced by input.
inputStructSeriesrequired
predicateTagPredicaterequired
StructTimeShiftSeries#
inputStructSeriesrequired
durationDurationConstantrequired
StructToStartOfIntervalSeries#
Snaps each point’s timestamp to the start of its enclosing fixed-size time bucket. Buckets are epoch-aligned: a point at timestamp t (nanoseconds) is rewritten to floor(t / window) * window. Values and tag groupings are preserved; aggregation of points that collide on the snapped timestamp is the responsibility of downstream nodes.
inputStructSeriesrequired
windowcompute.api.Durationrequired
Subtract#
Subtracts right from left pointwise.
leftNumericSeriesrequiredLeft-hand operand (minuend).
rightNumericSeriesrequiredRight-hand operand (subtrahend).
SuccessResult#
An empty JSON object, {}.
Summation#
The sum of points inside the time window.
An empty JSON object, {}.
SumSeries#
For every timestamp specified in the input series, outputs a value that is the sum for that timestamp across all input series. Only outputs timestamps where all input series have an entry for that timestamp, subject to interpolation.
inputsarray ofNumericSeriesInput series across which to take the pointwise sum
interpolationConfigurationInterpolationConfigurationdeprecatedDefaults to forward fill interpolation with a 1s interpolation radius Deprecated: Setting interpolation on this node is deprecated.
TagAnd#
Conjunction of two tag predicates (both must hold).
leftTagPredicaterequired
rightTagPredicaterequired
TagByIntervalsSeries#
Injects tags derived from interval boundaries onto a numeric series. Points within an interval get tagged; points outside all intervals are dropped. Intervals come from either events or ranges. If intervals with different tag map overlap, a data point falling in multiple intervals is duplicated into each matching output group. Overlapping intervals with equal tag maps are not allowed. Optional time shift subtracts a per-interval anchor from timestamps after matching.
inputNumericSeriesrequired
intervalsIntervalSourcerequired
timeShiftTagByIntervalsTimeShiftWhen set, shift each matched point’s timestamp by the chosen anchor for that interval.
TagByIntervalsTimeShift#
How to shift timestamps after tagging (subtract anchor from each point in the interval slice).
Set type to the name of one of the other fields, and set only that field.
intervalStartIntervalStartTimeShift
intervalEndIntervalEndTimeShift
eventTimeShiftEventTimeShift
TagFilter#
A filter on a single tag key and set of values. For an IN operator, the tag key must match at least one of the values in the set. For a NOT_IN operator, it must not match any of the values in the set.
keyStringConstantrequired
valuesarray ofStringConstant
operatorTagFilterOperatorrequired
TagFilterOperator#
IN
NOT_IN
TagFilters#
A set of tag filters to apply to a channel. Allows for combining filters on multiple tag keys, but throws if any one tag key is specified in more than one filter.
Set type to the name of one of the other fields, and set only that field.
singleTagFilter
andarray ofTagFilters
TaggedDataset#
Injects one ephemeral tag (key and value) onto the underlying dataset.
inputapi.DatasetrequiredThe underlying dataset to tag.
keyStringConstantrequiredTag key to inject onto each series in the dataset.
valueStringConstantrequiredTag value to assign to the injected key.
TagIn#
Constrains a tag key to be in the given set of values.
keyStringConstantrequired
valuesStringSetConstantV2required
TagNotIn#
Constrains a tag key to not be in the given set of values.
keyStringConstantrequired
valuesStringSetConstantV2required
TagPredicate#
Conjunctive tag predicate for dataset filtering. Compiles to resolved TagFilters for storage and resolution-time folding against known tag assignments (e.g. TaggedDataset, search branches).
Set type to the name of one of the other fields, and set only that field.
Tan#
Computes the trigonometric tangent of each value (input in radians).
xNumericSeriesrequiredInput numeric series (radians).
TemporalUnit#
HOURS
DAYS
WEEKS
Threshold#
Set type to the name of one of the other fields, and set only that field.
absoluteAbsoluteThreshold
percentagePercentageThreshold
ThresholdingRanges#
Produces a list of ranges for which the threshold condition is satisfied.
inputNumericSeriesrequired
thresholdDoubleConstantrequired
operatorThresholdOperatorrequired
persistenceWindowConfigurationPersistenceWindowConfiguration
ThresholdOperator#
GREATER_THAN
GREATER_THAN_OR_EQUAL_TO
LESS_THAN
LESS_THAN_OR_EQUAL_TO
EQUAL_TO
NOT_EQUAL_TO
ThresholdRangesComparison#
Selects the comparison applied to a numeric threshold.
Set type to the name of one of the other fields, and set only that field.
ThresholdRangesEqualityComparison#
An equality comparison against a numeric threshold, within an optional tolerance.
thresholdDoubleConstantrequiredThe value each input point is compared against.
toleranceDoubleConstantMaximum absolute difference from the threshold still counted as equal. Defaults to zero, i.e. exact equality.
ThresholdRangesOrderedComparison#
An ordering comparison against a single numeric threshold.
thresholdDoubleConstantrequiredThe value each input point is compared against.
ThresholdRangesV2#
Produces ranges where a numeric input satisfies a threshold comparison.
inputNumericSeriesrequired
minPointsIntegerConstantMinimum number of matching points. Defaults to 1.
minDurationcompute.api.DurationMinimum matching duration. Defaults to one nanosecond.
rangeStartRangeStartSelects whether the output begins at the first match or after persistence is satisfied.
comparisonThresholdRangesComparisonrequired
TimeDifferenceSeries#
Outputs a new series where each value is the difference between the time of the current and previous points.
TimeRangeFilter#
searches for files that intersect with the specified range.
startUtcTimestamp
endUtcTimestamp
TimescaleConnectionDetails#
databasestring required
hoststring required
portinteger required
usernamestring requiredThis is also the reference to the secret containing the password
passwordSecretRidstring (RID) requiredSecret Rid of password secret stored in Secrets Service.
TimescaleScrapingFilter#
Set type to the name of one of the other fields, and set only that field.
notTimescaleScrapingFilter
matchTablearray of stringMatch any of the specified tables in the database.
matchSchemaarray of stringMatch any of the specified schemas in the database.
TimeSeriesCurveFit#
Fits a curve to a series vs its timestamps.
seriesNumericSeriesrequiredThe series to fit. Timestamps will be used as x values and data as y values. The leftmost (earliest) timestamp will be used as the value of 0, and all other timestamps will be relative to that.
optionsTimeSeriesFitOptionsrequired
TimeSeriesFitOptions#
timeUnitTimeUnitTime unit used for the timestamps to fit against. Defaults to seconds if not specified.
TimeShiftedDataset#
Transforms a dataset by adding a time offset to all its data.
inputapi.DatasetrequiredThe underlying dataset to time-shift.
offsetcompute.api.DurationrequiredThe time shift to apply. Positive values shift data forward in time, negative values shift backward.
Timestamp#
Deprecated for new APIs. Use google.protobuf.Timestamp for absolute instants in proto APIs.
This legacy nanosecond precision timestamp type is represented by an epoch time in seconds and a nanosecond offset. The nanosecond offset is from the start of the epoch second, so must be less than 1 billion.
secondsinteger (int64) required
nanosinteger (int64) required
TimestampConstant#
A timestamp constant that can be a literal value or a named variable reference.
Set type to the name of one of the other fields, and set only that field.
literalTimestampCreate from a literal timestamp value.
variablestringCreate from a named reference.
TimestampMetadata#
seriesNamestring required
timestampTypecatalog.TimestampTyperequired
TimestreamChannelNameComponent#
Set type to the name of one of the other fields, and set only that field.
TimestreamConnectionDetails#
nominalRoleArnstring requiredThe role ARN owned by Nominal that has permission to assume the externalRoleArn.
nominalRoleRegionstring requiredAWS region that contains the Nominal role.
externalRegionstring requiredAWS region that contains the Timestream instance.
externalRoleArnstring requiredThe role ARN used to make queries to Timestream. This role is owned by the AWS account that owns the Timestream instance.
databasestring requiredDatabase inside of the Timestream instance to connect to.
TimestreamScrapingConfig#
filterarray ofTimestreamScrapingFilterIn order for data to be picked up by the scraper, it must match all filters in this list.
channelNameComponentsarray ofTimestreamChannelNameComponentchannelNameComponents will be combined, together with separator, to form a fully qualified channel name.
separatorstring required
maxLookBackPeriodHoursintegerThe maximum time in hours to look back over series data to find unique measure and tag combinations. If not specified, the default look back period will be used.
TimestreamScrapingFilter#
Set type to the name of one of the other fields, and set only that field.
matchTablearray of stringMatch any of the specified tables in the database.
TimeUnit#
DAYS
HOURS
MINUTES
SECONDS
MILLISECONDS
MICROSECONDS
NANOSECONDS
TypedPropertyValue#
Typed metadata value. Additional variants may be added as property types expand.
Set type to the name of one of the other fields, and set only that field.
stringValuestring
numericValuenumber
UdfSource#
Source provider for a user-defined function.
Set type to the name of one of the other fields, and set only that field.
rustRustUdfSource
UnaryArithmeticOperation#
COSCalculates the trigonometric cosine (input in radians)
SINCalculates the trigonometric sine (input in radians)
TANCalculates the trigonometric tangent (input in radians)
ABSCalculates the absolute value
ASINCalculates the arcsine, returning radians
ACOSCalculates the arccosine, returning radians
LOGCalculates the base 10 logarithm
LNCalculates the natural logarithm (base e)
SQRTCalculates the square root
UnaryArithmeticSeries#
Applies a point-wise transformation to a series. The transformation function is applied to every individual data point.
inputNumericSeriesrequired
operationUnaryArithmeticOperationrequired
UnboundedBehavior#
Dictates how empty range bounds should be handled by the compute node. Defaults to INFINITY.
INFINITY
WINDOW_BOUND
UnboundedRangeBehavior#
How to handle ranges with a missing start or end.
Set type to the name of one of the other fields, and set only that field.
Union#
Uses the union of all input timestamps as the alignment grid.
An empty JSON object, {}.
UnionRanges#
inputsarray ofRangeSeriesThe ranges to union.
UnitConversionSeries#
Convert the given series to a different unit.
inputNumericSeriesrequired
outputUnitstring required
UnitOverrideSeries#
Override the unit of the given series without changing its values. Unlike unit conversion, no compatibility with the input’s current unit is required.
inputNumericSeriesrequired
unitstringThe unit to report for the input series. If absent, the input’s unit is cleared.
UploadedAtFilter#
Searches for files whose uploadedAt falls within the specified range using half-open bounds [start, end). If start is omitted the range is unbounded below. If end is omitted the range is unbounded above.
startUtcTimestamp
endUtcTimestamp
UtcTimestamp#
secondsSinceEpochinteger (int64) required
ValueDifferenceSeries#
Calculates the first discrete difference of the input series. For each point, computes the difference between the current value and the previous value (current - previous). The first point in the input series is omitted from the output since it has no previous point to compare against.
This is equivalent to pandas DataFrame.diff() with period=1. Useful for analyzing changes between consecutive data points in time series data, such as detecting value increases or decreases.
Example: For input values [1.0, 1.0, 2.0, 2.5, 1.8], the output would be [0.0, 1.0, 0.5, -0.7] at the corresponding timestamps (the first timestamp is omitted).
inputNumericSeriesrequired
negativeValuesConfigurationNegativeValueConfigurationDefines the strategy for handling negative output values. Defaults to allowNegativeValues if not specified.
ValueMapSeries#
Maps a continuous numeric series to a discrete enum series using the specified value ranges.
inputNumericSeriesrequiredThe input series to map to an enumerated series
mappingarray ofRangeMapThe output of the first capturing range will be used. Ranges are start inclusive, end exclusive, must not overlap, and increasing from lowest to highest. Ranges can be open ended to the edge of the next or prior range. The first range can be open ended to negative infinity, and the last range can be open ended to positive infinity.
defaultStringConstantThe default value if not captured by any range. If not specified, points will be filtered.
VariableValue#
Set type to the name of one of the other fields, and set only that field.
doublenumber
computeNodeComputeNodeWithContext
durationrun.api.Duration
integerinteger
channelChannelSeriesdeprecatedDeprecated: use series instead.
derivedDerivedSeries
seriesSeriesVariable
stringstring
stringSetarray of string
timestampTimestamp
VideoFileMetadata#
Metadata specific to video files.
timestampManifestVideoFileTimestampManifestrequiredSpecifies how to determine absolute timestamps for each frame in the video. Can be an embedded MCAP manifest, an external sidecar file, or calculated from a starting offset applied to presentation timestamps.
segmentMetadataVideoSegmentsMetadataCached aggregate metadata about the segments comprising this video file after segmentation has completed. Includes frame counts, duration, and frame rate. Empty until segmentation is complete. Note: Min/max timestamps are stored in DatasetFile.bounds field. Raw file size is stored in DatasetFile.fileSizeBytes.
VideoFileTimestampManifest#
A video file’s timestamp manifest specifies how to determine absolute timestamps for each frame. In an MCAP file, this data is embedded within the source file. Alternatively, the manifest may be provided as an external sidecar file. If no manifest is available, the timestamps are calculated by applying a starting offset to the presentation timestamp of each frame.
Set type to the name of one of the other fields, and set only that field.
mcapMcapTimestampManifest
s3pathstring
s3pathsarray of stringMultiple sidecar manifest files whose per-frame timestamps are concatenated in list order. Used when a video’s frame timestamps are recorded across several files.
noManifestNoTimestampManifest
VideoSegmentsMetadata#
Segment metadata for video files. Lightweight version that excludes bounds and RID since those are stored elsewhere.
numFramesinteger requiredTotal number of frames across all segments.
numSegmentsinteger requiredNumber of segments the video was split into.
scaleFactornumber requiredScale factor applied to timestamps during segmentation.
mediaDurationSecondsnumber requiredTotal duration of the video in seconds.
mediaFrameRatenumber requiredAverage frame rate (FPS) calculated as total frames / duration.
VisualCrossingAvailableSeries#
Deprecated and not used for anything anymore, see record VisualCrossingAvailableSeries instead.
channelstring required
unitstring
descriptionstring
typeVisualCrossingTyperequired
VisualCrossingConnectionDetails#
apiKeySecretRidstring (RID) requiredSecret Rid of API key stored in Secrets Service.
VisualCrossingScrapingConfig#
availableSeriesarray ofVisualCrossingAvailableSeriesdeprecatedlist of available channels Deprecated: No longer used for anything
VisualCrossingType#
DOUBLE
STRING
WeakTimestampType#
If a CSV dataset is still being split, the timestamp type is not known.
ABSOLUTE
RELATIVE
PENDING
Window#
Set type to the name of one of the other fields, and set only that field.
durationDurationConstant
WithSeriesDataset#
Adds one or more derived series to the base dataset. Each entry’s definition must reference only the wrapping input. Series names must be unique within the request; a derived name replaces a same-named series from input.
inputapi.DatasetrequiredThe base dataset to which the new series will be added.
seriesarray ofNamedSeriesThe named derived series to add.
ZscoreSeries#
Computes the z-score for each series within a group-by family. For each group at each timestamp, the output is (value - mean) / stddev, where mean and stddev are computed across all groups in the family at that timestamp, using the interpolation configuration to align values across groups. When stddev is zero at a timestamp (e.g., all groups have the same value), that data point is excluded from the output. If groupByTags is empty, all groups in the input series will be considered as a single family. This node only produces meaningful results when the input has a group by.
inputNumericSeriesrequired
groupByTagsarray ofStringConstantTags to group by for the normalization. If left empty, all groups in the input will be considered as a single family. Otherwise, inputs will be normalized separately across the groups specified by the tags.
interpolationConfigurationInterpolationConfigurationdeprecatedConfiguration for aligning values across groups at each timestamp. Defaults to forward fill interpolation with a 1s interpolation radius. Deprecated: Setting interpolation on this node is deprecated.
Page design based on Scalar, MIT License.