Pipelines

A pipeline is a YAML document in your repository. This page describes what that document contains; the pipeline installation page uses it to deploy Aetheus itself.

Where the definition lives

Pipeline definitions are read from the repository, under .pipeline/<slug>.yaml. The backend re-reads the file from the canonical repository at every launch, so the definition on the default branch is always the one that runs. Editing a pipeline through the web UI writes the YAML back to the repository and commits it, which means a local checkout should be fetched before pushing by hand.

Skeleton

name: my-application-deploy
trigger: manual

variables:
  APP_VERSION: "1.0.$(BUILD_PIPELINE_RUNNUMBER)"
  DEPLOY_ROOT: "/srv/my-application"

stages:
  - name: Build
    os: linux
    steps:
      - name: Compile
        checkout: true
        working_directory: "$(WORKSPACE)"
        timeout_seconds: 1800
        shell: |
          set -eu
          ./build.sh

  - name: Deploy
    os: linux
    depends_on:
      - Build
    steps:
      - name: Publish to the host
        shell: sh deploy/publish.sh

Stages

A stage is a unit of scheduling. It is dispatched as a whole to one eligible agent, and it declares what it needs:

• os - linux or windows, used to select an agent.
• depends_on - a list of stage names. Stages with no unmet dependency run in parallel; this list is the only thing that serialises them.
• group - a display grouping in the run view, such as Deploy. It has no scheduling effect.
• artifacts - the paths this stage publishes once its steps succeed. Later stages, and later runs, restore them by name.
• execution_role - an optional routing constraint when agents are specialised.

Steps

Steps inside a stage run in order, on the same agent and in the same workspace. The common fields are:

Native step types

Some work is better expressed as a first-class step than as a script, because the backend then understands the result instead of parsing text. The types used by the pipelines shipped in the repository are:

Run parameters

A pipeline can ask for values at launch time through a top-level parameters: block. Each parameter has a type of string, boolean or choice, and a default so that a scheduled or webhook-triggered run still has a value.

parameters:
  - name: revision
    display_name: Revision recorded in the deployment journal
    display_name_fr: Révision consignée dans le journal de déploiement
    description: Recorded as is; a scheduled run records "scheduled".
    description_fr: Consignée telle quelle ; un run planifié consigne « scheduled ».
    type: string
    default: scheduled
    required: false

display_name_fr and description_fr are optional: the launch dialog shows them when the interface is in French, and falls back to display_name and description otherwise.

Variables

Values are substituted with $(NAME). Three sources feed the substitution:

variables:
  APP_VERSION: "1.0.$(BUILD_PIPELINE_RUNNUMBER)"

variable_libraries:
  - my-public-values

vaults:
  - my-application-secrets

A variable library is only visible to a pipeline that lists it; nothing is applied implicitly. A missing vault is a hard failure at variable resolution, so a step never silently receives an empty secret.

Built-in variables include $(WORKSPACE), the checkout directory, BUILD_SOURCEVERSION, the exact revision the run pinned, and BUILD_PIPELINE_RUNNUMBER, the per-pipeline build counter.

Building another repository

By default the workspace is the repository the definition lives in. A source: block checks out another repository of the same project instead, while the definition and its .pipeline/configs files keep coming from their own repository. It is how a pipeline kept in a private repository builds what a public mirror holds.

source:
  repository: my-public-mirror   # slug of a repository attached to the project
  branch: main                   # optional
  must_match_definition: true    # optional: refuse unless the trees are identical
  match_exclude_file: .pipeline/configs/public-source/exclude.txt

An external repository is attached to the project from its External repository tab; the token may be left empty for a public one. It is fetched when the run is prepared, then pinned to a commit. With must_match_definition, the run is refused unless the tree checked out is, file for file, the tree of the definition's revision, once the paths listed in the exclude file (one glob per line) are set aside. match_branch compares with the head of another branch of the definition's repository instead, for a public copy that is an adapted export recorded on that branch.

Templates

A pipeline can extend a published template with extends: <template>@<version> and supply only its own parameters. Templates are versioned and immutable: a published version cannot be edited, so adopting a change means moving to the next version explicitly. That is what keeps a fleet of pipelines from drifting under you.

Where to go next

• Install through an Aetheus pipeline - a minimal build, deploy and release pipeline.
• Concepts - artefacts, releases and environments.