Pipelines

A pipeline is a YAML document in your repository. This page describes what that document contains; the pipeline installation page uses it to deploy Aetheus itself.

Where the definition lives

Pipeline definitions are read from the repository, under .pipeline/<slug>.yaml. The backend re-reads the file from the canonical repository at every launch, so the definition on the default branch is always the one that runs. Editing a pipeline through the web UI writes the YAML back to the repository and commits it, which means a local checkout should be fetched before pushing by hand.

Skeleton

name: my-application-deploy
trigger: manual

variables:
  APP_VERSION: "1.0.$(BUILD_PIPELINE_RUNNUMBER)"
  DEPLOY_ROOT: "/srv/my-application"

stages:
  - name: Build
    os: linux
    steps:
      - name: Compile
        checkout: true
        working_directory: "$(WORKSPACE)"
        timeout_seconds: 1800
        shell: |
          set -eu
          ./build.sh

  - name: Deploy
    os: linux
    depends_on:
      - Build
    steps:
      - name: Publish to the host
        shell: sh deploy/publish.sh

Stages

A stage is a unit of scheduling. It is dispatched as a whole to one eligible agent, and it declares what it needs:

  • os - linux or windows, used to select an agent.
  • depends_on - a list of stage names. Stages with no unmet dependency run in parallel; this list is the only thing that serialises them.
  • group - a display grouping in the run view, such as Deploy. It has no scheduling effect.
  • artifacts - the paths this stage publishes once its steps succeed. Later stages, and later runs, restore them by name.
  • execution_role - an optional routing constraint when agents are specialised.

Steps

Steps inside a stage run in order, on the same agent and in the same workspace. The common fields are:

FieldMeaning
nameLabel shown in the run timeline and in the logs.
shellThe script to execute. A block scalar keeps multi-line scripts readable.
checkoutWhen true, clone the pinned source revision into the workspace before running.
working_directoryWhere to run, usually "$(WORKSPACE)".
timeout_secondsBudget for this step. Exceeding it fails the step rather than hanging the run.
continue_on_errorReport a failure as a warning and keep going, instead of failing the run.
typeSelects a native step instead of a shell script; see below.

Native step types

Some work is better expressed as a first-class step than as a script, because the backend then understands the result instead of parsing text. The types used by the pipelines shipped in the repository are:

TypeWhat it does
releaseRecords a release for this run: version, changelog, and whether it is deployed.
restore-artifactsRestores artefacts published by another stage or another run.
triggerLaunches another pipeline and, optionally, waits for it. The child run is shown nested in the parent's timeline.
coverage, lint, complexityIngest a report so it becomes a first-class result on the run and on the project.
scanner, analysis-gateRun a security or quality scanner, then block the run when findings cross a threshold.
smokeProbe a deployed URL and fail the stage when it does not answer as expected.

Run parameters

A pipeline can ask for values at launch time through a top-level parameters: block. Each parameter has a type of string, boolean or choice, and a default so that a scheduled or webhook-triggered run still has a value.

parameters:
  - name: revision
    display_name: Revision recorded in the deployment journal
    type: string
    default: scheduled
    required: false

Variables

Values are substituted with $(NAME). Three sources feed the substitution:

variables:
  APP_VERSION: "1.0.$(BUILD_PIPELINE_RUNNUMBER)"

variable_libraries:
  - my-public-values

vaults:
  - my-application-secrets

A variable library is only visible to a pipeline that lists it; nothing is applied implicitly. A missing vault is a hard failure at variable resolution, so a step never silently receives an empty secret.

Built-in variables include $(WORKSPACE), the checkout directory, BUILD_SOURCEVERSION, the exact revision the run pinned, and BUILD_PIPELINE_RUNNUMBER, the per-pipeline build counter.

Version numbers are build counters

BUILD_PIPELINE_RUNNUMBER increments on every launch of that pipeline, including failed and cancelled ones. A pipeline whose version is 1.1.$(BUILD_PIPELINE_RUNNUMBER) will therefore produce 1.1.57 on its 57th launch. If you need a semantic version, derive it from a tag rather than from this counter.

Templates

A pipeline can extend a published template with extends: <template>@<version> and supply only its own parameters. Templates are versioned and immutable: a published version cannot be edited, so adopting a change means moving to the next version explicitly. That is what keeps a fleet of pipelines from drifting under you.

Where to go next