# Training runs

> Watching queued work execute — statuses, tasks, and logs.

::div{class="flex items-start gap-6"}
::div{class="shrink-0 flex items-center justify-center mt-1 size-24 rounded-4xl bg-dim-200"}
:u-icon{name="i-lucide-play" class="size-16 text-primary"}
::

A **run** is the execution of a training request. Queueing an approved request creates a run; the run picks up the specified compute target, clones your code, executes the workflow stages, and publishes what it produced.
::

## Statuses

A run moves through a lifecycle:

| Status | Meaning |
| --- | --- |
| **Queued** | Created, waiting for a compute target to pick it up. |
| **Assigned** | Matched to a target; the agent hasn't claimed it yet. |
| **Acquired** | The agent has claimed it and is preparing — cloning, building the environment. |
| **In Progress** | Workflow stages are executing. |
| **Completed** | Every stage finished successfully. |
| **Failed** | A stage failed, or the run couldn't be prepared. |
| **Stopped** | Cancelled before it finished. |

**Completed**, **Failed**, and **Stopped** are terminal.

A run sitting in **Queued** for a long time usually means no target is available: a dedicated target whose agent is offline, or an on-demand target that can't launch — often an invalid provider credential. Check the compute target's status first.

## Tasks

Each workflow stage becomes a **task**, with its own status: **Pending**, **Running**, **Completed**, **Failed**, or **Skipped**. Tasks run in the order the workflow lists them, and when one fails the rest are **Skipped** rather than attempted.

This is the fastest way to read a failed run — find the failed task, and you know which stage broke.

```bash [TReqs CLI]
treqs jobs tasks <job-id>
```

## Logs

A run's log has more than one producer, which matters when you're debugging.

```bash [TReqs CLI]
treqs jobs logs <job-id>              # the workload's own output
treqs jobs logs <job-id> --follow     # keep polling until complete
treqs jobs logs <job-id> --agent      # what the agent recorded
treqs jobs logs <job-id> --task <id>  # narrow to one stage
```

`--agent` is the one people forget. If a task fails *before its command runs* — a missing interpreter, a checkout that didn't resolve, an environment that wouldn't build — the workload log is empty and the explanation is in the agent log.

To follow a run from the terminal:

```bash [TReqs CLI]
treqs jobs watch <job-id>   # lifecycle on stderr, logs on stdout
treqs jobs wait <job-id>    # block until terminal; non-zero unless Completed
```

`treqs jobs wait` exits non-zero unless the run reaches **Completed**, which makes it usable in a script. Ctrl-C during `watch` detaches without cancelling the run.

## Stopping a run

```bash [TReqs CLI]
treqs jobs stop <job-id>
treqs jobs cancel <job-id>  # `cancel` is just an alias for `stop`
```

A stopped run lands in **Stopped** and keeps its logs and task history.
