# Upload service The Upload Service manages file uploads to object storage. OpenAPI file: [`UploadService.json`](https://dev.nominal.io/api/_downloads/b6b7f41949a098461a761a90affb81a8/UploadService.json) ## Endpoints ### Initiate multipart upload POST `/upload/v1/multipart-upload` Initiates a multipart upload to object storage. Returns an uploadId that should be used with listParts, signPart, and completeMultipartUpload. Body `filename` string required : The desired name of the file in object storage. The final name will be prefixed with a timestamp to ensure uniqueness. `filetype` string required : The MIME type of the file, eg “text/csv” for CSVs or “application/octet-stream” for binary files. `workspace` string (RID) : The workspace in which to upload the file. If not provided, the file will be uploaded to the default workspace for the user’s organization, if that is configured and the user has access to it. Ignored if `resource` is provided. `resource` string (RID) : The RID of an existing resource (e.g. a dataset) that this upload is intended to feed. When provided, the server resolves the destination workspace from the resource and authorizes the caller via the resource’s permissions rather than via the workspace default-fallback. Use this in flows where the caller already knows the target resource (e.g. mesh file ingest into an existing remote dataset); this avoids the `Security:WorkspaceNotSpecified` failure for API keys whose user has no default workspace. If both `resource` and `workspace` are set, `resource` takes precedence. `destination` [`UploadDestination`](https://dev.nominal.io/api/rest/upload-service.md#uploaddestination) : Where the upload should be written. Defaults to UPLOADS. Whichever destination is chosen, the returned `bucket` must be passed back on the listParts/signPart/completeMultipartUpload/abortMultipartUpload calls. Values of `UploadDestination` `UPLOADS` : ​ `FILE_STORE` : ​ Response `uploadId` string : ​ `key` string : Identifies the uploaded object within its destination. For a FILE_STORE upload this is the value to pass as `UploadedObjectRef.objectKey` when committing a file revision. `bucket` string : Opaque handle for the upload’s destination. Pass it back unchanged on every subsequent call for this upload; do not parse or display it. Request **curl** ```shell curl -X POST "https://api.gov.nominal.io/api/upload/v1/multipart-upload" \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ --data '{ "filename": "", "filetype": "" }' ``` **requests** ```python import requests response = requests.post( "https://api.gov.nominal.io/api/upload/v1/multipart-upload", headers={"Authorization": "Bearer "}, json={"filename": "", "filetype": ""}, ) response.raise_for_status() print(response.json()) ``` **httpx** ```python import httpx response = httpx.post( "https://api.gov.nominal.io/api/upload/v1/multipart-upload", headers={"Authorization": "Bearer "}, json={"filename": "", "filetype": ""}, ) response.raise_for_status() print(response.json()) ``` **aiohttp** ```python import asyncio import aiohttp async def main() -> None: async with aiohttp.ClientSession() as session: async with session.post( "https://api.gov.nominal.io/api/upload/v1/multipart-upload", headers={"Authorization": "Bearer "}, json={"filename": "", "filetype": ""}, ) as response: response.raise_for_status() print(await response.json()) asyncio.run(main()) ``` Response: `200` ```json { "uploadId": "", "key": "", "bucket": "" } ``` ### List parts GET `/upload/v1/multipart-upload/{uploadId}` Lists the parts that have been uploaded for a given uploadId. Path parameters `uploadId` string required : ​ Query parameters `key` string required : ​ `bucket` string : The opaque destination handle returned as `bucket` from InitiateMultipartUploadResponse, passed back unchanged. Omit only for an upload opened without a destination. Response The body is an array of [`PartWithSize`](https://dev.nominal.io/api/rest/upload-service.md#partwithsize). Fields of `PartWithSize` `partNumber` integer : ​ `etag` string : ​ `size` integer (int64) : ​ Request **curl** ```shell curl "https://api.gov.nominal.io/api/upload/v1/multipart-upload/?key=" \ -H "Authorization: Bearer " ``` **requests** ```python import requests response = requests.get( "https://api.gov.nominal.io/api/upload/v1/multipart-upload/?key=", headers={"Authorization": "Bearer "}, ) response.raise_for_status() print(response.json()) ``` **httpx** ```python import httpx response = httpx.get( "https://api.gov.nominal.io/api/upload/v1/multipart-upload/?key=", headers={"Authorization": "Bearer "}, ) response.raise_for_status() print(response.json()) ``` **aiohttp** ```python import asyncio import aiohttp async def main() -> None: async with aiohttp.ClientSession() as session: async with session.get( "https://api.gov.nominal.io/api/upload/v1/multipart-upload/?key=", headers={"Authorization": "Bearer "}, ) as response: response.raise_for_status() print(await response.json()) asyncio.run(main()) ``` Response: `200` ```json [ { "partNumber": 1, "etag": "", "size": 1 } ] ``` ### Sign part POST `/upload/v1/multipart-upload/{uploadId}` Signs an upload request for a single part. Returns a URL that will execute the upload without further authentication. Path parameters `uploadId` string required : ​ Query parameters `key` string required : ​ `partNumber` integer required : ​ `bucket` string : The opaque destination handle returned as `bucket` from InitiateMultipartUploadResponse, passed back unchanged. Omit only for an upload opened without a destination. Response `url` string : ​ `headers` map of string to string : ​ `expiresInSeconds` integer (int64) : How long the signed URL stays valid, from when it was issued. Relative rather than absolute so a client with a skewed clock still bounds the request correctly; clients should time the upload out at this point instead of letting a request run past the signature’s lifetime and fail as an opaque connection reset. Request **curl** ```shell curl -X POST "https://api.gov.nominal.io/api/upload/v1/multipart-upload/?key=&partNumber=1" \ -H "Authorization: Bearer " ``` **requests** ```python import requests response = requests.post( "https://api.gov.nominal.io/api/upload/v1/multipart-upload/?key=&partNumber=1", headers={"Authorization": "Bearer "}, ) response.raise_for_status() print(response.json()) ``` **httpx** ```python import httpx response = httpx.post( "https://api.gov.nominal.io/api/upload/v1/multipart-upload/?key=&partNumber=1", headers={"Authorization": "Bearer "}, ) response.raise_for_status() print(response.json()) ``` **aiohttp** ```python import asyncio import aiohttp async def main() -> None: async with aiohttp.ClientSession() as session: async with session.post( "https://api.gov.nominal.io/api/upload/v1/multipart-upload/?key=&partNumber=1", headers={"Authorization": "Bearer "}, ) as response: response.raise_for_status() print(await response.json()) asyncio.run(main()) ``` Response: `200` ```json { "url": "", "headers": { "": "" }, "expiresInSeconds": 1 } ``` ### Complete multipart upload POST `/upload/v1/multipart-upload/{uploadId}/complete` Completes a multipart upload to object storage. This should be called after all parts have been uploaded. Will throw EmptyMultipartUpload if there are 0 parts. Path parameters `uploadId` string required : ​ Query parameters `key` string required : ​ `bucket` string : The opaque destination handle returned as `bucket` from InitiateMultipartUploadResponse, passed back unchanged. Omit only for an upload opened without a destination. Body The body is an array of [`Part`](https://dev.nominal.io/api/rest/upload-service.md#part). Fields of `Part` `partNumber` integer required : ​ `etag` string required : ​ Response `location` string : ​ Request **curl** ```shell curl -X POST "https://api.gov.nominal.io/api/upload/v1/multipart-upload//complete?key=" \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ --data '[ { "partNumber": 1, "etag": "" } ]' ``` **requests** ```python import requests response = requests.post( "https://api.gov.nominal.io/api/upload/v1/multipart-upload//complete?key=", headers={"Authorization": "Bearer "}, json=[{"partNumber": 1, "etag": ""}], ) response.raise_for_status() print(response.json()) ``` **httpx** ```python import httpx response = httpx.post( "https://api.gov.nominal.io/api/upload/v1/multipart-upload//complete?key=", headers={"Authorization": "Bearer "}, json=[{"partNumber": 1, "etag": ""}], ) response.raise_for_status() print(response.json()) ``` **aiohttp** ```python import asyncio import aiohttp async def main() -> None: async with aiohttp.ClientSession() as session: async with session.post( "https://api.gov.nominal.io/api/upload/v1/multipart-upload//complete?key=", headers={"Authorization": "Bearer "}, json=[{"partNumber": 1, "etag": ""}], ) as response: response.raise_for_status() print(await response.json()) asyncio.run(main()) ``` Response: `200` ```json { "location": "" } ``` ### Abort multipart upload POST `/upload/v1/multipart-upload/{uploadId}/abort` Aborts a multipart upload to S3. Frees storage used by previously uploaded parts and prevents further uploads to the same uploadId. Path parameters `uploadId` string required : ​ Query parameters `key` string required : ​ `bucket` string : The opaque destination handle returned as `bucket` from InitiateMultipartUploadResponse, passed back unchanged. Omit only for an upload opened without a destination. Response No content (HTTP 204). Request **curl** ```shell curl -X POST "https://api.gov.nominal.io/api/upload/v1/multipart-upload//abort?key=" \ -H "Authorization: Bearer " ``` **requests** ```python import requests response = requests.post( "https://api.gov.nominal.io/api/upload/v1/multipart-upload//abort?key=", headers={"Authorization": "Bearer "}, ) response.raise_for_status() ``` **httpx** ```python import httpx response = httpx.post( "https://api.gov.nominal.io/api/upload/v1/multipart-upload//abort?key=", headers={"Authorization": "Bearer "}, ) response.raise_for_status() ``` **aiohttp** ```python import asyncio import aiohttp async def main() -> None: async with aiohttp.ClientSession() as session: async with session.post( "https://api.gov.nominal.io/api/upload/v1/multipart-upload//abort?key=", headers={"Authorization": "Bearer "}, ) as response: response.raise_for_status() asyncio.run(main()) ``` Response: `204` No content. ### Sign download POST `/upload/v1/sign-download` Returns a short-lived signed GET URL for an object already in the uploads bucket. Use this to hand a freshly-uploaded object’s path to an external service (e.g. Dagger) without giving that service any Nominal credentials. The caller must be authorized to upload to the workspace that owns the path (encoded as the first segment of the object key). Body `path` string required : Path of an object in the Nominal uploads bucket, in the form returned by CompleteMultipartUploadResponse.location. The caller must be authorized to read the workspace whose UUID prefixes the object key. Response `url` string : Short-lived signed GET URL. Expires after 15 minutes. `expiresAt` string (date-time) : Absolute expiry timestamp. Callers must stop using the URL after this. Request **curl** ```shell curl -X POST "https://api.gov.nominal.io/api/upload/v1/sign-download" \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ --data '{ "path": "" }' ``` **requests** ```python import requests response = requests.post( "https://api.gov.nominal.io/api/upload/v1/sign-download", headers={"Authorization": "Bearer "}, json={"path": ""}, ) response.raise_for_status() print(response.json()) ``` **httpx** ```python import httpx response = httpx.post( "https://api.gov.nominal.io/api/upload/v1/sign-download", headers={"Authorization": "Bearer "}, json={"path": ""}, ) response.raise_for_status() print(response.json()) ``` **aiohttp** ```python import asyncio import aiohttp async def main() -> None: async with aiohttp.ClientSession() as session: async with session.post( "https://api.gov.nominal.io/api/upload/v1/sign-download", headers={"Authorization": "Bearer "}, json={"path": ""}, ) as response: response.raise_for_status() print(await response.json()) asyncio.run(main()) ``` Response: `200` ```json { "url": "", "expiresAt": "2026-01-01T00:00:00Z" } ``` ### Upload file POST `/upload/v1/upload-file` Uploads a file to S3. Intended for smaller files. Query parameters `fileName` string required : ​ `sizeBytes` integer (int64) : Specifies the size in bytes of the file being uploaded. This is required to be set if the file is greater than 100MB, however in practice this should probably always be set. `workspace` string (RID) : The workspace in which to upload the file. If not provided, the file will be uploaded to the default workspace for the user’s organization, if that is configured and the user has access to it. Body: `application/octet-stream` The body is the raw bytes. Response The body is a string. Request **curl** ```shell curl -X POST "https://api.gov.nominal.io/api/upload/v1/upload-file?fileName=" \ -H "Authorization: Bearer " \ -H "Content-Type: application/octet-stream" \ --data-binary "@" ``` **requests** ```python from pathlib import Path import requests response = requests.post( "https://api.gov.nominal.io/api/upload/v1/upload-file?fileName=", headers={ "Authorization": "Bearer ", "Content-Type": "application/octet-stream", }, data=Path("").read_bytes(), ) response.raise_for_status() print(response.json()) ``` **httpx** ```python from pathlib import Path import httpx response = httpx.post( "https://api.gov.nominal.io/api/upload/v1/upload-file?fileName=", headers={ "Authorization": "Bearer ", "Content-Type": "application/octet-stream", }, content=Path("").read_bytes(), ) response.raise_for_status() print(response.json()) ``` **aiohttp** ```python import asyncio from pathlib import Path import aiohttp async def main() -> None: async with aiohttp.ClientSession() as session: async with session.post( "https://api.gov.nominal.io/api/upload/v1/upload-file?fileName=", headers={ "Authorization": "Bearer ", "Content-Type": "application/octet-stream", }, data=Path("").read_bytes(), ) as response: response.raise_for_status() print(await response.json()) asyncio.run(main()) ``` Response: `200` ```json "" ``` ## Errors [Errors](https://dev.nominal.io/api/legacy.md#errors) shows how these errors are sent. Each error’s `errorName` is `UploadService:` followed by its name below. | Status | Name | Notes | | --- | --- | --- | | 400 | `EmptyMultipartUpload` | | | 400 | `IncorrectSizeBytes` | | | 400 | `NullKey` | | | 400 | `SignDownloadPathOutsideUploadsBucket` | | | 400 | `UploadBucketNotAllowed` | A multipart-upload follow-up call specified a `bucket` this service does not recognize. It must be the value returned as `bucket` by initiateMultipartUpload, unchanged. | | 400 | `WorkspaceResourceMismatch` | Both `workspace` and `resource` were set on the upload request, but the workspace derived from the resource does not match the explicitly-supplied workspace. Returned as a defensive sanity check so callers don’t accidentally believe they’re uploading to one workspace while the file is actually being routed to another. | | 403 | `SignDownloadUnauthorized` | | | 404 | `ResourceNotFoundOrUnauthorized` | The resource referenced by an upload’s `resource` field was not found or the caller is not authorized to access it. Returned as NOT_FOUND (rather than PERMISSION_DENIED) to avoid leaking existence information about resources the caller cannot see. | | 404 | `UploadObjectNotFound` | A multipart-upload follow-up call named a File Store object the ledger has no row for. A malformed key and an unknown one are reported identically, so a caller cannot tell which of the two it supplied. | Each error’s `errorName` is `IngestService:` followed by its name below. | Status | Name | Notes | | --- | --- | --- | | 400 | `BatchGranularityMismatch` | All dataset granularities must match. | | 400 | `ContainerizedExtractorTimestampMetadataMissing` | Neither the containerized extractor (or its active container image) nor the ingest request provides timestamp metadata, so the extractor output cannot be indexed. Register the image with default timestamp metadata or supply it on the request. | | 400 | `FailedToDownloadFileUserError` | | | 400 | `FileDeletionNotSupported` | The dataset file is part of a multi-file ingest job and cannot be deleted on its own. Delete the entire ingest job instead. | | 400 | `FrameCountTimestampsMismatch` | | | 400 | `IncompatibleConnectionTypeAndLogQuery` | | | 400 | `IncompatibleFileForDataset` | | | 400 | `IncompatibleInputs` | The inputs provided to the ingest job are incompatible with each other | | 400 | `IngestJobNotCompatibleForRerun` | The ingest job exists but is not compatible to be re-run. | | 400 | `InvalidBucket` | | | 400 | `InvalidCustomTimestampFormat` | | | 400 | `InvalidS3Path` | | | 400 | `InvalidUrl` | | | 400 | `MissingMetadataForReingest` | The timestamp metadata required to reingest the dataset is missing and must be provided in the request. | | 400 | `ParallelReingestIntoDatasetV1` | Cannot ingest multiple datasets in parallel into older datasets. It is recommended to reingest all source datasets and the target dataset into a new dataset destination. | | 400 | `RecursiveReingest` | The target dataset is included in the source datasets. | | 400 | `ReingestTooManyFiles` | The number of files to reingest is too large. | | 400 | `RunBoundsInverted` | | | 400 | `UnsupportedIngestSource` | | | 400 | `UnsupportedRequestTypeForReIngest` | | | 403 | `ContainerizedExtractorUnauthorizedPasswordAccess` | The user is not authorized to access the containerized extractor password. | | 404 | `DatasetFileNotFound` | The target dataset file could not be found. | | 404 | `DatasetNotFound` | The target dataset could not be found. | | 404 | `IngestJobNotFound` | | | 404 | `IngestTransformNotFound` | The ingest transform does not exist, or its parent ingest job is not visible to the caller. Returned as NOT_FOUND (rather than PERMISSION_DENIED) to avoid leaking existence information about transforms the caller cannot see. | | 404 | `S3PathNotFound` | | | 404 | `StreamingSessionNotFound` | The streaming session could not be found. | | 409 | `IngestJobNotCancellable` | The ingest job exists but is in a terminal state (COMPLETED, FAILED, or CANCELLED) and therefore cannot be cancelled. | | 500 | `ContainerizedExtractorContainerFailed` | | | 500 | `FailedToDownloadFile` | | | 500 | `IngestJobMissingDatasetForIdempotencyKey` | A prior ingest job already exists for this idempotency key but never produced a dataset (the original submit did not get far enough). The idempotency key deduplicates to that existing job and does not re-run ingestion; rerun the job identified by ingestJobRid to complete it. | ## Types ### Part `partNumber` integer required : ​ `etag` string required : ​ ### PartWithSize `partNumber` integer required : ​ `etag` string required : ​ `size` integer (int64) required : ​ ### UploadDestination The object-storage destination an upload should be written to. `UPLOADS` : ​ `FILE_STORE` : ​ Page design based on [Scalar](https://scalar.com), MIT License.