# Export jobs (gRPC)

Write query results to files in object storage, and download them when the export finishes. An export runs in the background, so you start it, poll it, and then download its files.

The service is `ExportJobService` in the `nominal.exportjobs.v1` proto package. It has three calls:

| Call | Purpose |
| --- | --- |
| `CreateExportJob` | Validate and authorize the request, then start the export. |
| `GetExportJob` | Return one export, with a download URL for each file once it is complete. |
| `ListExportJobs` | List a workspace’s exports, newest first. |

Connect and authenticate as described in the [gRPC API](https://dev.nominal.io/api/grpc.md) overview. Every call needs view access to the export’s workspace. A call without access gets the same `NOT_FOUND` as a call for an export that doesn’t exist.

## Start an export

`CreateExportJob` takes a workspace, one source, and optional settings.

| Field | Type | Required | Notes |
| --- | --- | --- | --- |
| `workspace_rid` | string | Yes | The workspace the export runs in. |
| `sql` | `SqlExportSource` | One source is required | `query`: the `SELECT` statement, up to 100,000 characters. |
| `compute` | `ComputeExportSource` | One source is required | `definition`: the same export definition `DataExportService.CreateExport` takes. |
| `zip` | `ZipExportSource` | One source is required | `selections`: 1 to 100 `DatasetFilesSelection` messages. |
| `settings` | `ExportSettings` | No | Must be unset for a `zip` source. See [Settings](https://dev.nominal.io/api/export-jobs.md#settings). |

`sql`, `compute` and `zip` are alternatives: set exactly one.

The response holds an `ExportJob`. Keep its `export_id`, which is the id to pass to `GetExportJob`. The call returns as soon as the export is started, so `status` is usually `EXPORT_JOB_STATUS_SUBMITTED` or `EXPORT_JOB_STATUS_IN_PROGRESS`, though a very fast export can already be complete.

An SQL export, shown as JSON:

```json
{
  "workspace_rid": "<workspace-rid>",
  "sql": {
    "query": "SELECT ts, value FROM points_double WHERE dataset_rid = '<dataset-rid>' ORDER BY ts"
  },
  "settings": {
    "file_format": "EXPORT_FILE_FORMAT_PARQUET"
  }
}
```

A zip export of every file in one dataset, and of two files in another:

```json
{
  "workspace_rid": "<workspace-rid>",
  "zip": {
    "selections": [
      { "dataset_rid": "<dataset-rid>", "all_files": {} },
      { "dataset_rid": "<other-dataset-rid>", "file_ids": { "ids": ["<file-uuid>", "<file-uuid>"] } }
    ]
  }
}
```

### Sources

- **SQL**: runs the query as the [SQL export endpoint](https://dev.nominal.io/sql-reference/rest-api/export.md) does. The caller needs the access that endpoint requires.
- **Compute**: exports computed channel data. The caller needs view access to the workspace and access to the data the definition reads. The server checks the definition again when the export runs.
- **Zip**: puts dataset files in one zip. Each `DatasetFilesSelection` names a `dataset_rid` and exactly one of `file_ids` (1 to 1,000 file UUIDs, zipped in the order given), `query` (every uploaded, non-deleted file that matches a dataset file query) or `all_files` (every uploaded, non-deleted file). The caller needs read access to every dataset named. The selections are resolved when the export runs, not when you create it.

### Settings

`settings` is the state of an export form. The server stores it and returns it as given, so a client can prefill a new export from an earlier one.

- For an `sql` export, the server uses only `file_format` and `parquet_options`. `file_format` is `EXPORT_FILE_FORMAT_CSV` (also the default when unset) or `EXPORT_FILE_FORMAT_PARQUET`. `parquet_options` is valid only with Parquet.
- For a `compute` export, the server uses none of the settings. The `definition` decides what is exported, and isn’t checked against `settings`.
- A `zip` export takes no settings, and a request that sets them is rejected with `INVALID_ARGUMENT`.

## Get an export

`GetExportJob` takes the `export_id` and returns an `ExportJob`:

| Field | Notes |
| --- | --- |
| `export_id` | The id to pass to `GetExportJob`. |
| `settings` | The settings sent at creation, if any. |
| `status` | An `ExportJobStatus`. See [Export lifecycle](https://dev.nominal.io/api/export-jobs.md#export-lifecycle). |
| `files` | The exported files. Set only when `status` is `EXPORT_JOB_STATUS_COMPLETED`. |
| `create_time` | When the export was requested. |

Each `ExportFile` has a `file_name`, `size_bytes`, `expire_time`, and, from `GetExportJob` only, a `download_url` and its `download_url_expire_time`. The URL is a presigned `GET` URL that carries its own credentials, so treat it as a secret and don’t log it. Each call to `GetExportJob` returns a new URL. When one expires, call it again.

An export can be seen by the user who created it, and by other users who can view the workspace once the export is complete and they can read every dataset it read. An export that is still running is visible only to its creator.

## List exports

`ListExportJobs` returns the exports in a workspace that you can see, newest first.

| Field | Type | Required | Notes |
| --- | --- | --- | --- |
| `workspace_rid` | string | Yes | The workspace to list. A workspace you can’t view lists nothing. |
| `page_size` | integer | No | 1 to 1,000. Defaults to 50. |
| `page_token` | string | No | The `next_page_token` of the previous response. |

The response has `export_jobs` and, when there are more, `next_page_token`. Each entry is an `ExportJobSummary` with the `export_id`, `status` and `create_time`, but not the settings or files. Call `GetExportJob` for those. A page can hold fewer than `page_size` exports when some are left out because you can’t read their datasets, so continue until `next_page_token` is unset.

## Export lifecycle

Poll `GetExportJob` until the status is `EXPORT_JOB_STATUS_COMPLETED`, `EXPORT_JOB_STATUS_FAILED` or `EXPORT_JOB_STATUS_CANCELLED`.

| Status | Meaning |
| --- | --- |
| `EXPORT_JOB_STATUS_SUBMITTED` | Recorded, but not yet submitted to the job service. |
| `EXPORT_JOB_STATUS_IN_PROGRESS` | Running. |
| `EXPORT_JOB_STATUS_COMPLETED` | Done. `files` is set. |
| `EXPORT_JOB_STATUS_FAILED` | Failed. |
| `EXPORT_JOB_STATUS_CANCELLED` | Cancelled. |
| `EXPORT_JOB_STATUS_EXPIRED` | Completed, but its files are past their retention period and no longer available. |

Files are kept for 7 days after the export completes, after which the export reports `EXPORT_JOB_STATUS_EXPIRED`.

## Errors

A failed call returns a gRPC status. For the errors this service defines, the status has a `google.rpc.ErrorInfo` detail whose `reason` is one of these, as described in [Errors](https://dev.nominal.io/api/grpc.md#errors):

| `reason` | Status code | When |
| --- | --- | --- |
| `EXPORT_JOB_ERROR_NOT_FOUND` | `NOT_FOUND` | The export doesn’t exist, or you can’t see it. The two look the same. |
| `EXPORT_JOB_ERROR_INVALID_PAGE_TOKEN` | `INVALID_ARGUMENT` | `page_token` isn’t one `ListExportJobs` returned. |
| `EXPORT_JOB_ERROR_EXPORT_NOT_CONFIGURED` | `FAILED_PRECONDITION` | The deployment has no object storage for exports. |
| `EXPORT_JOB_ERROR_TOO_MANY_FILES` | `INVALID_ARGUMENT` | A zip request names more files than one export may. |
| `EXPORT_JOB_ERROR_DATASET_FILES_NOT_FOUND` | `NOT_FOUND` | A zip request names a dataset or file that is missing, in another workspace, deleted, or unreadable by you. These look the same. |

`CreateExportJob` also returns `NOT_FOUND` when the workspace rid is malformed or you can’t view the workspace, and `INVALID_ARGUMENT` for a request that fails validation, such as a missing source or a `zip` request with `settings`.
