Nominal Drives#

Store, version and download files in workspace-scoped drives with nominal.experimental.drives. This API is experimental and may change between releases.

A managed drive stores files in Nominal. A virtual drive mirrors an external provider and is read-only through Nominal. Both expose files through drive-relative paths.

Connect the client#

This guide assumes you have installed the client and setup your API keys.

Create or retrieve a drive#

Use create_drive() to create a managed drive in the client’s workspace:

from nominal.experimental import drives

drive = drives.create_drive(client, "telemetry", description="Raw telemetry from test stands")

Retrieve an existing drive by its workspace-unique ID with get_drive_by_id(), or by RID with get_drive():

Creation, lookup by ID and listing accept workspace_rid to select a workspace explicitly. A drive owned by another resource may have no ID; use its RID to retrieve it.

Upload files and new versions#

put_file() uploads a local file. A string destination creates a new file and must be unoccupied:

readings = drive.put_file("local/readings.csv", "raw/readings.csv")
print(readings.rid, readings.path, readings.size_bytes)

To add a version to an existing file, pass the file as the destination. Its RID stays the same:

readings = drive.put_file("local/readings-corrected.csv", readings)

put_file_obj() accepts a seekable binary stream and uploads from its current position to its end:

import io

generated = drive.put_file_obj(io.BytesIO(b"time,value\n0,1\n"), "raw/generated.csv")

The destination determines the stored filename and content type. The service checks the destination after the upload completes. If the path is occupied or the destination’s revision has changed, the call raises NominalFileStoreError. The passed destination snapshot is unchanged; use the returned file.

List and retrieve files#

list_files() lists immediate children by default:

for entry in drive.list_files("raw"):
    if isinstance(entry, drives.DriveDirectory):
        print("directory:", entry.path)
    elif isinstance(entry, drives.ManagedDriveFile):
        print("file:", entry.path, entry.size_bytes, entry.rid)
    elif isinstance(entry, drives.VirtualDriveFile):
        print("provider file:", entry.path, entry.size_bytes, entry.provider)

For a managed drive, pass recursive=True to collect active files at every depth, without directory entries:

for file in drive.list_files("raw", recursive=True):
    print(file.path)

Recursive listing cannot include removed files. For an immediate-child listing, include_removed=True includes them. All listing modes collect every page.

Use get_file() when you know the path:

file = drive.get_file("raw/readings.csv")

Paths are drive-relative, with no leading or trailing slash and no . or .. segments. The empty path denotes the root: drive.list_files() lists its immediate children.

Download content#

download() writes the snapshot’s content into an existing directory, using the basename of its drive path. It refuses to overwrite an existing file:

local_path = file.download("downloads/")

DriveFileRevision downloads the same way, using that revision’s content and filename. Files and revisions expose DownloadAvailability: AVAILABLE, BYTES_RECLAIMED, NO_CONTENT, or UNKNOWN. Availability is a snapshot; a later download can still fail if content is reclaimed or the provider becomes unavailable.

Move files and directories#

move_to() refreshes the file in place:

readings.move_to("archive/readings.csv")

A string destination must be free. Passing a managed file or revision as the destination requires that revision to still be current. Moving onto another file removes that destination file and puts the moved file at its path.

move_directory() moves all active files beneath a directory:

drive.move_directory("raw", "processed")

The destination must be empty and cannot contain or sit inside the source. The service can reject a move that exceeds its limits. Existing file snapshots keep their old paths until refresh() is called; the directory move returns no per-file details.

Remove and restore files#

remove() removes a file while keeping its history:

generated.refresh()  # Pick up its revision after the directory move.
generated.remove()

revisions() returns revisions newest first, including removal markers. Restore a content revision with restore():

revision = next(r for r in generated.revisions() if r.state is drives.DriveFileState.ACTIVE)
restored = revision.restore("raw/generated.csv")

If the owning file is still active, pass that file or its current revision as the destination. If it was removed, a free path can be used. Restoration requires the revision’s content to remain available.

Apply changes in a batch#

Use batch() to put, move, remove, restore, or move directories in one request. Results follow the input order. By default, each change succeeds or fails independently and later changes still run after a failure:

pending = drive.get_file("incoming/report.csv")
assert isinstance(pending, drives.ManagedDriveFile)
batch = drive.batch()
batch.move(pending, "reports/report.csv")
batch.move_directory("incoming/calibration", "reports/calibration")
results = batch.submit()

for result in results:
    if isinstance(result, drives.FileChangeFailure):
        print(result.code, result.message)
    elif isinstance(result, drives.FileChangeSuccess):
        print(result.file.path, result.revision.rid)
    else:
        print("Directory moved")

Use batch.remove(file) to remove a file or batch.restore(revision, destination) to restore a revision. Inputs must belong to this drive. Batch submission leaves your input snapshots unchanged; use returned files or refresh existing snapshots. A batch is single-use: calling submit consumes it even if submission fails, and subsequent additions or submissions raise RuntimeError. Reconcile an uncertain outcome before creating a replacement batch.

To add uploads, prepare each put before submitting the batch. Preparation uploads bytes and pins the destination’s revision precondition, without creating or changing a drive file:

prepared = drive.prepare_put_file("local/summary.csv", "reports/summary.csv")
batch = drive.batch()
batch.put(prepared)
batch.move_directory("staging", "published")
results = batch.submit(failure_policy=drives.FailurePolicy.ABORT)

ABORT applies every change or none. The first failed change reports its error; the other results report BATCH_FAILURE. Uploads happen before the transaction, so an aborted batch does not undo uploaded bytes. Use prepare_put_file_obj for a seekable binary stream.

CONTINUE, the default, accepts up to 1,000 changes. ABORT accepts up to 100. The client does not split batches. Earlier changes are visible to later changes, but all revision preconditions come from the snapshots you submit. Changing the same file twice can therefore make the second change fail its revision precondition. Request-level errors raise Nominal exceptions; see Recover after a transport error before retrying.

Read virtual drives#

VirtualDrive exposes provider-backed files through get_file and nonrecursive list_files. Check provider connectivity with status():

status = virtual_drive.status()
print(status.state, status.message, status.last_successful_check_at)

A VirtualDriveFile keeps the revision reference observed when it was retrieved. Downloads use that reference directly; Nominal resolves it and may stream content from the provider. Use resolve() when you need a durable revision RID. Resolving does not copy or guarantee retention of the bytes.

Azure Blob files can be listed and identified. Resolving and downloading their content depends on server support; the current service reports those operations as unimplemented.

Handle rejected changes#

A rejected single-file operation raises NominalFileStoreError with a FileStoreErrorCode:

try:
    drive.put_file("local/readings.csv", "raw/readings.csv")
except drives.NominalFileStoreError as error:
    if error.code is drives.FileStoreErrorCode.PATH_ALREADY_EXISTS:
        print("Choose a new path or retrieve the file to add a version.")
    else:
        raise

Failures reported as gRPC statuses use the client’s general exceptions, such as NominalNotFoundError. Revision conflicts are surfaced to the caller rather than retried with a newer revision.

If a single-operation response has a missing or unexpected outcome, the client raises ValueError without refreshing the input snapshot or replaying the request. The server may still have applied the change; reconcile its state before retrying.

Recover after a transport error#

The client does not automatically replay file changes after a transport error such as UNAVAILABLE or RESOURCE_EXHAUSTED. This applies to DriveBatch.submit and the single-operation methods put_file, put_file_obj, move_to, remove, restore, and move_directory. Reads and multipart byte uploads keep their existing retry behavior. For a put, the upload and the final change that adds a file revision are separate requests; only the latter has this no-retry policy.

A transport error can arrive after the server has applied the changes. For example, a directory move can succeed but its response can be lost. Replaying it would then fail because the source directory no longer exists, hiding the original success.

A transport exception is different from a returned FileChangeFailure:

  • With CONTINUE, some or all changes may already have applied when the response is lost.

  • With ABORT, the batch still applies all or none, but a transport exception does not tell you which happened. It is not evidence of rollback. A received batch response containing failures, including BATCH_FAILURE, does establish that the atomic batch applied nothing.

Before submitting changes again:

  1. Refresh managed files you already hold. file.refresh() looks up the stable file RID, so it follows moves and returns the removed state after a removal. Inspect revision history with file.revisions() when the current state alone is insufficient.

  2. For new files or directory moves, inspect the expected destination and source paths with get_file or list_files. Another writer may have changed them, so path existence alone is not proof that your request succeeded. Compare identities, revisions, and expected state.

  3. Submit only changes you have established are still needed. Preserve the original revision preconditions unless you deliberately decide to act on newer state. If the outcome remains unclear, stop and investigate instead of replaying the batch.

Do not wrap these writes in an unconditional retry loop. Refreshing an input and retrying without inspecting it can replace the original precondition with a newer revision and apply a change you did not intend. The same caution applies to prepared puts: their destination precondition remains pinned to preparation time, and an aborted batch does not undo the upload.

Ingest a stored revision#

Use IngestBuilder to ingest a managed file revision into a dataset without downloading or uploading it again. See Ingest files already in a Drive for a batch combining a Drive revision with a local file.