CI/CD Pipelines

E2Engine can run end-to-end tests as part of a CI/CD pipeline.

A typical pipeline starts the system under test, creates the required E2Engine resources, runs a TestSuite, and uses the resulting execution status to determine whether the pipeline should pass or fail.

This guide shows this workflow using Docker and GitHub Actions.

CI workflow

A CI job using E2Engine typically follows this sequence:

Build system
    │
    ▼
Start test environment
    │
    ▼
Create E2Engine resources
    │
    ▼
Run TestSuite
    │
    ▼
Check TestSuiteExecution
    │
    ├── passed ──▶ CI succeeds
    │
    └── failed/error ──▶ CI fails

The important distinction is between running a test suite and checking its result.

run testsuite creates an execution and returns its ID:

EXECUTION_ID="$(
    e2engine run testsuite smoke payment-demo -q
)"

The execution may still be scheduled or running at this point.

Use check testsuiteexecution to wait for the execution to finish and translate its final status into a process exit code:

e2engine check testsuiteexecution "${EXECUTION_ID}" -q

If the execution passes, the command exits successfully. If it finishes with failed or error, or does not finish before the configured timeout, the command exits with a non-zero status.

This makes check suitable as the final assertion of a CI job.

Running E2Engine with Docker

E2Engine can run as a short-lived Docker container for each CLI operation.

For example:

docker run \
    --rm \
    --network e2engine-network \
    -v "${DATA_DIR}:/data" \
    -v "${ROOT_DIR}/e2engine:/specs:ro" \
    -e E2ENGINE_DATA_DIR=/data \
    ghcr.io/e2engine/cli:0.1.0 \
    get environments

Each invocation starts a new E2Engine container.

The persistent data directory is mounted into every invocation:

host .e2engine/
       │
       ▼
container /data

This allows separate CLI invocations to operate on the same E2Engine state.

The specifications are mounted read-only:

repository e2engine/
       │
       ▼
container /specs

For a more detailed explanation of running E2Engine with Docker, see Running E2Engine with Docker.

Docker network

The E2Engine container and the application services must be able to communicate in both directions.

Create a dedicated Docker network:

docker network create e2engine-network

Then start the application services on that network. The public E2Engine demo uses a real Account service and a Payment API as the system under test:

docker run \
    -d \
    --name account \
    --network e2engine-network \
    e2engine-demo-account:local

docker run \
    -d \
    --name payment-api \
    --network e2engine-network \
    -e ACCOUNT_SERVICE_ADDR=e2engine:8082 \
    -e NOTIFICATION_SERVICE_ADDR=e2engine:8083 \
    -e FRAUD_SERVICE_URL=http://e2engine:8081 \
    e2engine-demo-payment-api:local

Docker container names can be used as hostnames by other containers on the same user-defined network.

In this example, all downstream calls made by the Payment API go through E2Engine:

Payment API
    │
    ├── HTTP ──▶ e2engine:8081 ─────────────▶ mocked Fraud service
    │
    ├── gRPC ──▶ e2engine:8082 ──▶ account:50051
    │                                      real Account service
    │
    └── gRPC ──▶ e2engine:8083 ─────────────▶ mocked Notification service

The Environment defines both the E2Engine service boundary and, for a real service, the downstream target:

spec:
  services:
    - id: account
      kind: grpc
      mode: real
      address: 0.0.0.0:8082
      grpc_target: account:50051

The address is where E2Engine accepts and observes calls from the system under test. grpc_target identifies the real service to which E2Engine forwards those calls.

This routing is important for call verification. If the Payment API called account:50051 directly, the application call could succeed, but E2Engine would not observe it and an expectation such as calls.account.count: 1 would report zero calls.

Mocked services also use E2Engine service boundaries, but have no downstream target. E2Engine handles those calls from their configured fixtures.

Because the Payment API reaches these boundaries using the hostname e2engine, the E2Engine container that executes the test suite must run on the same network with the stable name e2engine.

Creating resources

Once the application services are running, create the E2Engine Environment:

ENVIRONMENT_ID="$(
    e2engine create environment /specs/payment-demo.docker.env.yml -q
)"

Create the Tests:

e2engine create test /specs/successful-payment.docker.test.yml -q
e2engine create test /specs/account-rejection.docker.test.yml -q
e2engine create test /specs/fraud-rejection.docker.test.yml -q

Then create the TestSuite:

TEST_SUITE_ID="$(
    e2engine create testsuite /specs/smoke.ts.yml -q
)"

Quiet mode is useful in automation because commands that create resources print only the created resource ID:

-q

This makes IDs easy to capture in shell variables.

Running the TestSuite

Run the TestSuite against the Environment:

EXECUTION_ID="$(
    e2engine run testsuite \
        "${TEST_SUITE_ID}" \
        "${ENVIRONMENT_ID}" \
        -q
)"

The returned value is the ID of the newly created TestSuiteExecution.

Running the suite and determining its final result are intentionally separate operations.

The execution can be inspected independently:

e2engine get testsuiteexecution "${EXECUTION_ID}"

For CI, use check instead.

Checking the result

Check the TestSuiteExecution:

e2engine check testsuiteexecution "${EXECUTION_ID}" -q

If the execution is still scheduled or running, E2Engine waits for it to reach a terminal status.

The command succeeds when the final status is:

passed

and returns a non-zero exit code when the execution finishes with:

failed
error

A timeout also causes the command to fail.

Because normal shell scripts and CI systems already understand process exit codes, no JSON or YAML parsing is required.

For example:

set -euo pipefail

EXECUTION_ID="$(
    e2engine run testsuite smoke payment-demo -q
)"

if ! e2engine check testsuiteexecution "${EXECUTION_ID}" -q; then
    echo "Test suite failed."
    e2engine get testsuiteexecution "${EXECUTION_ID}"
    exit 1
fi

echo "E2E tests passed."

Using an explicit if around check allows the script to print the TestSuiteExecution details before failing the CI job.

Execution check timeout

By default, check uses the configured execution check timeout.

It can be configured using:

runtime:
  execution_check_timeout: 30s

or the corresponding environment variable:

E2ENGINE_RUNTIME_EXECUTION_CHECK_TIMEOUT=30s

The timeout can also be overridden for a particular command:

e2engine check testsuiteexecution "${EXECUTION_ID}" \
    --timeout 2m \
    -q

This can be useful when CI environments require a longer timeout than local development.

A reusable CI script

Rather than putting the complete test environment setup directly into a CI provider configuration, keep the workflow in the repository.

For example:

e2engine-demo/
├── account/
├── payment-api/
├── proto/
├── e2engine/
├── scripts/
│   └── demo-ci.sh
├── Makefile
└── .github/
    └── workflows/
        └── ci.yml

The script can own the complete E2E lifecycle:

scripts/demo-ci.sh
    │
    ├── build Linux application binaries
    ├── build application images
    ├── create Docker network
    ├── start application containers
    ├── create E2Engine resources
    ├── run TestSuite
    ├── check TestSuiteExecution
    └── clean up

A simplified version based on the public E2Engine demo looks like this:

#!/usr/bin/env bash

set -euo pipefail

ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"

NETWORK="e2engine-network"
DATA_DIR="${ROOT_DIR}/.e2engine"

E2ENGINE_VERSION="${E2ENGINE_VERSION:-0.1.0}"
E2ENGINE_IMAGE="ghcr.io/e2engine/cli:${E2ENGINE_VERSION}"

ACCOUNT_BIN="${ROOT_DIR}/account/bin/account"
PAYMENT_BIN="${ROOT_DIR}/payment-api/bin/payment-api"

ACCOUNT_SERVICE_NAME="account"
PAYMENT_API_NAME="payment-api"

ACCOUNT_SERVICE_IMAGE="e2engine-demo-account:local"
PAYMENT_API_IMAGE="e2engine-demo-payment-api:local"

cleanup() {
    docker rm -f \
        "${PAYMENT_API_NAME}" \
        "${ACCOUNT_SERVICE_NAME}" \
        >/dev/null 2>&1 || true

    docker network rm "${NETWORK}" >/dev/null 2>&1 || true

    rm -rf "${DATA_DIR}"

    rm -f \
        "${ACCOUNT_BIN}" \
        "${PAYMENT_BIN}"
}

trap cleanup EXIT

cd "${ROOT_DIR}"

echo "Building demo services..."

mkdir -p "$(dirname "${ACCOUNT_BIN}")"
mkdir -p "$(dirname "${PAYMENT_BIN}")"

DOCKER_ARCH="$(docker version --format '{{.Server.Arch}}')"

case "${DOCKER_ARCH}" in
    amd64|arm64)
        ;;
    *)
        echo "Unsupported Docker architecture: ${DOCKER_ARCH}"
        exit 1
        ;;
esac

CGO_ENABLED=0 GOOS=linux GOARCH="${DOCKER_ARCH}" \
    go build -o "${ACCOUNT_BIN}" ./account/cmd/main.go

CGO_ENABLED=0 GOOS=linux GOARCH="${DOCKER_ARCH}" \
    go build -o "${PAYMENT_BIN}" ./payment-api/cmd/main.go

docker buildx build \
    --platform "linux/${DOCKER_ARCH}" \
    --load \
    -t "${ACCOUNT_SERVICE_IMAGE}" \
    ./account

docker buildx build \
    --platform "linux/${DOCKER_ARCH}" \
    --load \
    -t "${PAYMENT_API_IMAGE}" \
    ./payment-api

echo "Creating Docker network..."

docker network create "${NETWORK}" >/dev/null

echo "Starting account service..."

docker run \
    -d \
    --name "${ACCOUNT_SERVICE_NAME}" \
    --network "${NETWORK}" \
    "${ACCOUNT_SERVICE_IMAGE}" \
    >/dev/null

echo "Starting payment API..."

docker run \
    -d \
    --name "${PAYMENT_API_NAME}" \
    --network "${NETWORK}" \
    -e ACCOUNT_SERVICE_ADDR=e2engine:8082 \
    -e NOTIFICATION_SERVICE_ADDR=e2engine:8083 \
    -e FRAUD_SERVICE_URL=http://e2engine:8081 \
    "${PAYMENT_API_IMAGE}" \
    >/dev/null

mkdir -p "${DATA_DIR}"

run_e2engine() {
    docker run \
        --rm \
        --name e2engine \
        --network "${NETWORK}" \
        -v "${DATA_DIR}:/data" \
        -v "${ROOT_DIR}/e2engine:/specs:ro" \
        -v "${ROOT_DIR}/proto:/proto:ro" \
        -e E2ENGINE_DATA_DIR=/data \
        "${E2ENGINE_IMAGE}" \
        "$@"
}

echo "Creating E2Engine environment..."

run_e2engine create env /specs/payment-demo.docker.env.yml

echo "Creating E2Engine tests..."

run_e2engine create test /specs/successful-payment.docker.test.yml
run_e2engine create test /specs/account-rejection.docker.test.yml
run_e2engine create test /specs/fraud-rejection.docker.test.yml

echo "Creating E2Engine test suite..."

run_e2engine create ts /specs/smoke.ts.yml

echo "Running test suite..."

EXECUTION_ID=$(run_e2engine run ts smoke payment-demo -q)

if ! run_e2engine check tse "${EXECUTION_ID}" -q; then
    echo "Test suite failed."
    run_e2engine get tse "${EXECUTION_ID}"
    exit 1
fi

echo "Test suite executions:"

run_e2engine get testsuiteexecutions

echo "Test executions:"

run_e2engine get testexecutions

echo "E2E tests passed."

This is the same lifecycle exercised by the public E2Engine demo. The exact application containers and E2Engine resources will differ between projects, but the overall structure remains the same.

The explicit GOOS=linux cross-compilation is important when this script is run from macOS: Docker Buildx selects the image platform, but it does not convert a host-built Mach-O executable into a Linux executable. The binary copied into a Linux container must itself be built for Linux.

The /proto mount is required by this example because the Environment references an external protobuf file. External files used by a resource must be available inside the E2Engine execution container at the path expected by the resource.

Make target

Expose the CI workflow through a simple Make target:

.PHONY: demo-ci
demo-ci:
  @./scripts/demo-ci.sh

The same command can now be used locally:

make demo-ci

and by the CI system.

This keeps the CI provider configuration small and makes the E2E workflow reproducible outside the CI environment.

GitHub Actions

A GitHub Actions workflow can delegate the E2E workflow to the same Make target:

name: ci

on:
  push:
  pull_request:
  workflow_dispatch:

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@v7

      - name: Setup Go
        uses: actions/setup-go@v7
        with:
          go-version: stable

      - name: Run CI
        run: make demo-ci

GitHub Actions is responsible only for providing the runner and invoking the repository’s CI entry point:

GitHub Actions
      │
      ▼
make demo-ci
      │
      ▼
scripts/demo-ci.sh
      │
      ├── Docker environment
      ├── application services
      ├── E2Engine
      └── TestSuiteExecution check

The E2E workflow itself remains independent of GitHub Actions.

The same make demo-ci command can be invoked from other CI systems such as GitLab CI, Jenkins, or a local development environment.

Cleanup

CI scripts should clean up containers, networks, generated binaries, and temporary E2Engine state even when a test fails.

A shell trap provides a simple way to guarantee cleanup:

cleanup() {
    docker rm -f \
        payment-api \
        account \
        >/dev/null 2>&1 || true

    docker network rm e2engine-network \
        >/dev/null 2>&1 || true

    rm -rf .e2engine

    rm -f \
        account/bin/account \
        payment-api/bin/payment-api
}

trap cleanup EXIT

Because the cleanup runs on script exit, resources are removed after both successful and failed executions.

TestExecution checks

The same mechanism can be used when a pipeline runs an individual Test rather than a TestSuite.

Run the Test:

EXECUTION_ID="$(
    e2engine run test \
        successful-payment \
        payment-demo \
        -q
)"

Then check the resulting TestExecution:

e2engine check testexecution "${EXECUTION_ID}" -q

The command uses the same exit-code semantics as check testsuiteexecution.

Summary

A portable E2Engine CI workflow can be kept deliberately simple:

CI provider
    │
    ▼
make demo-ci
    │
    ▼
repository demo-ci script
    │
    ├── build
    ├── start services
    ├── create E2Engine resources
    ├── run tests
    ├── check execution
    └── cleanup

Keep environment orchestration in the repository rather than embedding it deeply in a CI provider configuration.

Use run to create an execution and check to wait for its result and expose that result as a standard process exit code.

This keeps the same E2Engine workflow usable locally and across different CI/CD systems.