Workflows

Workflows live in the repo, not in TReqs. That's deliberate — the workflow is versioned with the code it runs, so a training request pinned to a commit gets exactly the workflow that existed at that commit.

Conventionally they live under .treqs/workflows/:

.treqs/workflows/train.yaml

Shape

Every key that isn't reserved is a stage. There is no steps: or jobs: wrapper — the top level is the list of stages, and they run in the order written.

name: train
working_directory: .
secrets:
  - WANDB_API_KEY

preprocess: python preprocess.py --input data.csv --output features.parquet
train: python train.py --data features.parquet --output model.pt
evaluate: python evaluate.py --model model.pt --output metrics.json

That workflow has three stages: preprocess, train, evaluate.

The reserved keys — everything that is not a stage — are:

Multi-command stages

A stage can be a list, in which case the commands run in sequence:

train:
  - python -m pip install -e .
  - python train.py --config configs/base.yaml

Python environment

The environment.python block asks TReqs to build a virtual environment before any stage runs:

environment:
  python:
    version: "3.11"
    venv_path: .venv
    install_roar: true
    requirements:
      - -r requirements.txt
    build_requirements:
      - setuptools
      - wheel

• version — the Python version, as major.minor.
• venv_path — where the virtual environment is created. Stages run with it active.
• install_roar — install roar into the environment so the run captures lineage. Set this if you want the run to publish to GLaaS.
• requirements — passed to pip install. Entries can be package specifiers or -r pointers to a requirements file in the repo.
• build_requirements — installed first, for packages that need build tooling present before the main install.

Secrets

secrets lists the names of secrets to pull from the compute target the run executes on. They arrive as environment variables:

secrets:
  - WANDB_API_KEY
  - HF_TOKEN

Only declared secrets are injected. Naming a secret the target doesn't have is a configuration error — set it first with treqs compute secrets set, described in Compute targets.

Never put secret values in the workflow file. It's in your repository; anyone who can read the repo can read it.

Validating

Check a workflow against a compute target before opening a training request around it:

treqs doctor --target gpu-box --workflow .treqs/workflows/train.yaml

treqs doctor validates the project binding, the git source, the workflow, and compute readiness together, which catches most of what would otherwise fail several minutes into a run.

How a run uses it

When a training request is queued, TReqs clones the repository at the pinned commit, reads the workflow from that commit, builds the environment if one is declared, and executes each stage in order as a task. A failing stage fails the run and later stages are skipped.

Stage-level status is visible while the run executes — see Training runs.