Inspecting results

E2Engine stores the result of every test and test suite execution.

Execution results are structured records containing the request that was executed, the response that was received, the expectations that were evaluated, observed service calls, and any deviations or execution errors.

Use these records to understand not only whether a test passed or failed, but what happened during execution.

Test results

A TestExecution identifies the Environment and Test that were executed and contains the resulting status and summary.

A completed execution has one of three terminal statuses:

Status	Meaning
passed	Execution completed and all expectations were satisfied.
failed	Execution completed, but one or more expectations were not satisfied.
error	The test could not be executed or evaluated normally.

The execution summary contains the details needed to understand the result:

TestExecution
│
├── request
├── response
├── expect
├── calls
├── deviations
└── error

Request and response

The request section records the request executed by E2Engine.

For an HTTP test:

request:
  http:
    method: POST
    url: http://127.0.0.1:8080/payments
    headers:
      Content-Type:
        - application/json
    body_json: '{"account_id":"acc-001","amount":12500}'

The response section records the response received during execution:

response:
  http:
    status_code: 201
    headers:
      Content-Type:
        - application/json
    body_json: '{"id":"payment-1","status":"completed"}'

For gRPC tests, the corresponding sections contain the service, method, metadata, message, and response status.

Expectations

The expect section records the expectations that were evaluated during the execution.

For example:

expect:
  http:
    status_code: 201
  calls:
    - service_id: fraud
      count: 1
      http:
        method: POST
        path: /check

Keeping the evaluated expectations in the result makes the execution self-describing: the result shows both what E2Engine observed and what it compared those observations against.

Observed calls

The calls section contains service interactions observed through the Environment while the test was running.

For example:

calls:
  - service_id: fraud
    http:
      request:
        method: POST
        path: /check
        body_json: '{"account_id":"acc-001","amount":12500}'
      response:
        status_code: 200
        body_json: '{"approved":true}'

Each call identifies the Environment service and contains the observed request and response.

For HTTP calls, this can include the method, path, query parameters, headers, request body, response status, headers, and response body.

For gRPC calls, the result contains the RPC, metadata, request message, response status, and response message.

Observed calls are useful both for verifying explicit call expectations and for understanding how the system behaved during a test.

Deviations

When an execution has status failed, inspect deviations first.

A deviation describes a difference between expected and observed behavior.

For example, an unexpected HTTP status can produce:

deviations:
  - field: status_code
    expected: "201"
    actual: "500"

A missing expected service interaction can produce:

deviations:
  - field: calls.fraud.count
    expected: "1"
    actual: "0"

Other deviations can identify response body differences or requests to mocked services that did not match any configured fixture.

A useful way to investigate a failed test is:

status: failed
      │
      ▼
  deviations
      │
      ├── response mismatch ──▶ inspect response
      │
      └── call mismatch ──────▶ inspect calls

The response, expect, and calls sections provide the surrounding execution data needed to understand each deviation.

Execution errors

An execution with status error is different from a failed assertion.

A failed execution completed normally but observed behavior did not satisfy the Test expectations.

An error means E2Engine could not execute or evaluate the test normally.

In this case, inspect summary.error:

status: error
summary:
  error: context deadline exceeded

Errors can result from problems such as transport failures, execution timeouts, invalid execution input, or runtime and evaluation errors.

A useful starting point is therefore:

status
  │
  ├── passed ──▶ no deviations
  │
  ├── failed ──▶ inspect deviations
  │
  └── error ───▶ inspect summary.error

Test suite results

A TestSuiteExecution contains the aggregate result of running the resolved Tests in a suite.

Its summary reports the number of Tests in each terminal state:

summary:
  total: 3
  passed: 2
  failed: 1
  errors: 0

The suite also contains its individual TestExecution records.

TestSuiteExecution
│
├── summary
│   ├── total
│   ├── passed
│   ├── failed
│   └── errors
│
└── tests
    ├── TestExecution A
    ├── TestExecution B
    └── TestExecution C

When a suite fails, use the suite summary to understand the overall result, then inspect the individual TestExecutions that did not pass.

For example:

status: failed
tests_count: 3
summary:
  total: 3
  passed: 2
  failed: 1
  errors: 0

indicates that one Test completed with unmet expectations. Its deviations, response, and calls provide the details.

If any child TestExecution ends with error, the suite itself ends with error.

A practical inspection flow

For an individual TestExecution:

Check status
     │
     ├── passed
     │     └── inspect response and calls if needed
     │
     ├── failed
     │     └── inspect deviations
     │             │
     │             ├── compare expect and response
     │             └── compare expected and observed calls
     │
     └── error
           └── inspect summary.error

For a TestSuiteExecution, start with the aggregate summary and then inspect the child TestExecutions that have status failed or error.

Because execution results preserve both expectations and observations, they can also be consumed programmatically by CI/CD pipelines and other automation.

See Executions for the execution data model, Running a test for the test execution flow, and Running a test suite for test suite execution.