Expressions & Data Flow

This document explains how data flows through workflows using inputs, outputs, variables, and expressions.

Table of Contents

  1. Overview
  2. Inputs
  3. Variables
  4. Outputs
  5. Expressions Syntax
  6. Context & Scope
  7. Common Patterns
  8. Type Safety
  9. Examples

Overview

Data Flow Architecture

text
1Workflow Inputs
2       ↓
3   Trigger Binding
4       ↓
5   Job Execution
6       ├── Access: inputs, vars, needs
7       ├── Produce: outputs
8       └── Execute: with input expressions
9       ↓
10   Downstream Jobs
11       ├── Access: needs.<job>.outputs
12       ├── Conditional: if expressions
13       └── Loop: matrix expansion

Three Types of Data

  1. Inputs — Values passed when workflow is triggered
  2. Variables — Static values defined in workflow
  3. Outputs — Values produced by jobs

Inputs

Declaring Inputs

Inputs are declared in trigger definitions:

yaml
1on:
2  workflow_dispatch:
3    inputs:
4      project_name:
5        type: string
6        required: true
7        description: Name of the project
8      budget:
9        type: number
10        required: false
11        defaultValue: 100000
12      include_testing:
13        type: boolean
14        required: false
15        defaultValue: true
16      priority_level:
17        type: string
18        enum: [low, medium, high, critical]
19        required: true
20      team_members:
21        type: array
22        required: false
23        itemType: string
24      config:
25        type: object
26        required: false

Input Types

TypeExampleNotes
string"hello"Scalar text value
number3.14Floating-point number
integer42Whole number
booleantruetrue or false
array["a", "b"]List of values (must specify itemType)
object{key: value}JSON object

Input String Formats

For string type inputs:

yaml
1inputs:
2  description:
3    type: string
4    format: multiline           # Renders as text area in UI

Input Defaults

Provide default values:

yaml
1inputs:
2  environment:
3    type: string
4    defaultValue: staging
5    required: false             # Becomes optional if default provided

Input Validation

Inputs can have constraints:

yaml
1inputs:
2  region:
3    type: string
4    enum: [us-west, us-east, eu-west]  # Closed set
5    required: true
6
7  priority:
8    type: integer
9    required: true

Invalid inputs are rejected at dispatch time with clear error messages.

Accessing Inputs

In workflow jobs, access inputs via expression:

yaml
1jobs:
2  process:
3    uses: xema/agent
4    with:
5      project: ${{ inputs.project_name }}
6      budget: ${{ inputs.budget }}
7      include_testing: ${{ inputs.include_testing }}
8      priority: ${{ inputs.priority_level }}

Example: User Input Validation

yaml
1on:
2  workflow_dispatch:
3    inputs:
4      scope:
5        type: string
6        enum: [simple, standard, enterprise]
7        required: true
8
9jobs:
10  start:
11    uses: xema/agent
12    with:
13      message: "Starting workflow for scope: ${{ inputs.scope }}"

If user tries to trigger with scope: advanced (not in enum), it's rejected before execution.


Variables

Declaring Variables

Static variables defined at workflow level:

yaml
1vars:
2  organization: "Acme Corp"
3  max_retries: 3
4  review_template: "enterprise"
5  documentation_url: "https://docs.acme.com"

Accessing Variables

In workflow jobs:

yaml
1jobs:
2  setup:
3    uses: xema/agent
4    with:
5      org: ${{ vars.organization }}
6      retries: ${{ vars.max_retries }}
7      template: ${{ vars.review_template }}

Advantages Over Hardcoding

  • Centralized — Single place to update
  • Reusable — Used across multiple jobs
  • Clear — Documents workflow configuration
  • Maintainable — No scattered magic values

Outputs

Activity outputs

Every activity returns a flat envelope of named outputs. Each declared output is an artifact reference (ArtifactRef) — never inline content. The reference shape is:

ts
1interface ArtifactRef {
2  artifactId: string;
3  versionId: string;
4  version: number;
5  hash: string;
6  type: string;     // markdown_doc, json_payload, external_blob, …
7  title?: string;
8}

See Output Envelope for the per-activity output catalogue.

Declaring job outputs

A job's outputs: block selects which activity outputs it re-exports under the job's own namespace. Each entry is an expression — most commonly forwarding an activity output as-is via job.outputs.<name>:

yaml
1jobs:
2  analyze:
3    uses: xema/agent
4    with:
5      agentRef: requirements
6      deliverableSpecRef: risk-assessment
7      agentContext:
8        prompt: Assess the request and produce the typed risk assessment.
9    outputs:
10      assessment: ${{ job.outputs.structuredOutput }}

Consuming outputs in downstream jobs

Downstream jobs read job outputs via needs.<job>.outputs.<name> and can reach into the ArtifactRef fields directly:

yaml
1jobs:
2  analyze:
3    uses: xema/agent
4    with:
5      agentRef: requirements
6      deliverableSpecRef: risk-assessment
7      agentContext:
8        prompt: Assess the request and produce the typed risk assessment.
9    outputs:
10      assessment: ${{ job.outputs.structuredOutput }}
11
12  publish:
13    needs: [analyze]
14    uses: customer/file-risk-[email protected]
15    with:
16      artifactId: ${{ needs.analyze.outputs.assessment.artifactId }}
17      versionId: ${{ needs.analyze.outputs.artifacts[0].versionId }}
18      version: ${{ needs.analyze.outputs.artifacts[0].version }}

xema/publish-kb accepts the (artifactId, versionId, version) triple and fetches the bytes from artifact-store — workflow authors never copy bytes between steps.

Payload Reach-In (Typed)

When an upstream job's output is governed by a deliverable spec, you can reach into the artifact's parsed payload directly from a workflow expression. The compiler validates the path against the bound spec's schema at compile time, so a typo on a payload field fails the workflow before any job dispatches.

yaml
1jobs:
2  audit:
3    uses: xema/agent
4    with:
5      agentRef: builder@3
6      deliverableSpecRef: metrics-[email protected]
7      agentContext:
8        prompt: Produce the current metrics snapshot.
9
10  notify:
11    needs: [audit]
12    uses: xema/publish-[email protected]
13    with:
14      space: ops
15      slug: coverage
16      message: "Coverage: ${{ needs.audit.outputs.report.coverage }}%"
17      # `coverage` is a top-level field on `[email protected]` —
18      # the compiler validated it; the runtime pre-fetches the payload
19      # before this expression evaluates.

How the compiler decides what's typed:

  1. Per-output binding from the upstream activity's outputBindings: block — strongest signal, wins when present.
  2. Falls back to the upstream job's with.deliverableSpecRef (for activities like the agent action whose spec is workflow-author-chosen).
  3. Neither set, AND the field isn't an ArtifactRef envelope field (artifactId, versionId, version, hash, type, title) → DSL_FIELD_NOT_TYPED at compile time. Use the fromJSON(...) escape hatch for ad-hoc payloads, or bind a spec.

Runtime spec-version assertion: the workflow runtime fetches each referenced payload before the expression evaluates, and asserts the producer's schemaVersion matches the spec version the compiler pinned. If a producer upgrades its emit to a newer spec version while consumers stay compiled against the older one, the runtime fails fast with a clear spec-drift error rather than feeding the consumer a payload shape it doesn't expect.

fromJSON / toJSON

For payloads without a bound spec, or for ad-hoc parsing of stringly-typed fields, the DSL exposes GitHub-Actions-style runtime accessors:

FunctionInputOutput
fromJSON(ref)An ArtifactRef from a job outputThe fetched, parsed payload
fromJSON(str)A JSON string literalStandard JSON parse result
toJSON(value)AnythingJSON-stringified representation
yaml
1# Runtime parse — works on ArtifactRef or a literal string
2- message: ${{ fromJSON(needs.draft.outputs.response).title }}
3- payload: ${{ toJSON(needs.draft.outputs.structured) }}

The compiler does NOT type-check fromJSON(...) arguments — it's the documented escape hatch when typed reach-in isn't available. Use typed reach-in (outputs.<name>.<field>) when a spec is bound; reach for fromJSON(...) only when one isn't.

Matrix Job Outputs

When a job has matrix expansion, outputs are available as arrays:

yaml
1jobs:
2  build-all:
3    strategy:
4      matrix:
5        os: [ubuntu, windows, macos]
6    uses: xema/agent
7    with:
8      os: ${{ matrix.os }}
9    outputs:
10      build_id: ${{ result.build_id }}
11
12  collect-builds:
13    needs: build-all
14    uses: xema/agent
15    with:
16      # Access all matrix outputs as array
17      build_ids: ${{ needs.build-all.outputs }}

Or use matrixGather to flatten:

yaml
1jobs:
2  process-all:
3    needs: build-all
4    uses: xema/agent
5    with:
6      all_build_ids: ${{ needs.build-all.outputs.build_id[*] }}

Workflow-Level Outputs

Export outputs from entire workflow (for parent workflows):

yaml
1on:
2  workflow_call:
3    outputs:
4      final_result:
5        value: ${{ needs.final-job.outputs.result }}
6      artifacts:
7        value: ${{ needs.final-job.outputs.artifacts }}

Expressions Syntax

Expression Basics

Expressions are template literals:

${{ <expression> }}

Literals

yaml
1${{ 'string' }}              # String literal
2${{ 42 }}                    # Number literal
3${{ true }}                  # Boolean literal
4${{ null }}                  # Null literal

Variable References

yaml
1${{ vars.organization }}
2${{ inputs.project_name }}
3${{ needs.job1.outputs.summary }}
4${{ matrix.os }}
5${{ dynamic.service }}
6${{ trigger.payload.repo }}

Operators

Comparison Operators

yaml
1${{ inputs.budget > 100000 }}
2${{ inputs.budget >= 100000 }}
3${{ inputs.budget < 50000 }}
4${{ inputs.budget <= 50000 }}
5${{ inputs.budget == 100000 }}
6${{ inputs.budget != 100000 }}

Logical Operators

yaml
1${{ inputs.deploy && inputs.production }}    # AND
2${{ inputs.draft || inputs.publish }}        # OR
3${{ !inputs.skip_tests }}                    # NOT

String Operators

yaml
1${{ 'prefix-' + vars.organization }}         # Concatenation
2${{ contains('hello world', 'world') }}      # Contains check

Functions

Conditional Functions

  • success() — Every direct need succeeded and no ancestor failed or was cancelled, and the run was not cancelled (a skipped need makes it false)
  • failure() — Any transitive ancestor job failed (including by timeout)
  • cancelled() — Cancellation of the run was requested
  • timedOut() — Any transitive ancestor job failed because its effective timeout elapsed
  • always() — Always true (run regardless)

timedOut() implies failure(): a timed-out job is a failed job. A job's effective timeout is its timeout: setting or, when it sets none, the workflow or action default, else the platform default of one hour; reaching it counts as a timeout. A heartbeat or liveness timeout is a plain failure (failureReason of error).

failure() and cancelled() are not mutually exclusive. If a job failed and the run was then cancelled, both are true, because the failure before the cancel is real.

yaml
1jobs:
2  test:
3    uses: xema/agent
4
5  deploy:
6    needs: test
7    if: ${{ success() }}         # Only if test succeeded
8
9  notify:
10    needs: [test, deploy]
11    if: ${{ failure() }}         # Only if any ancestor failed
12
13  notify-timeout:
14    needs: [test, deploy]
15    if: ${{ timedOut() }}        # Only if an ancestor hit its timeout
16
17  cleanup:
18    needs: [test, deploy]
19    if: ${{ always() }}          # Always run

Needs Fields

Each entry in needs.<job> exposes:

FieldMeaning
needs.<job>.outputs.<name>A declared output of the job
needs.<job>.outcomeThe terminal outcome the job settled at (for example failed)
needs.<job>.failureReason'timeout' or 'error' when the job failed, null otherwise

failureReason is a closed set: timeout means the job's effective timeout elapsed, and everything else is error.

yaml
1  report:
2    needs: [build]
3    if: ${{ always() && needs.build.failureReason == 'timeout' }}

String Functions

  • contains(string, substring) — Check substring
  • startsWith(string, prefix) — Check prefix
  • endsWith(string, suffix) — Check suffix
yaml
1jobs:
2  check-branch:
3    needs: previous
4    if: ${{ contains(trigger.payload.branch, 'release') }}
5    uses: xema/agent
6
7  check-version:
8    needs: previous
9    if: ${{ startsWith(vars.version, '2.') }}
10    uses: xema/agent

Array Functions

  • length(array) — Array size
  • first(array) — First element
  • last(array) — Last element
yaml
1jobs:
2  check-multiple:
3    if: ${{ length(needs.previous.outputs.items) > 0 }}
4    uses: xema/agent

Object & Array Access

Object Property Access

yaml
1# Dot notation
2${{ needs.job.outputs.person.name }}
3
4# Bracket notation (for keys with special chars)
5${{ needs.job.outputs['config-version'] }}

Array Element Access

yaml
1${{ needs.job.outputs.items[0] }}            # First element
2${{ needs.job.outputs.items[-1] }}           # Last element

Array Spread

yaml
1# All elements
2${{ needs.job.outputs.items[*] }}
3
4# Filter/map operations (not directly supported, use jobs instead)

Escape Characters

Escape special characters:

yaml
1${{ 'string with $ and { special' }}         # In strings

For shell commands:

yaml
1with:
2  command: 'echo ${{ inputs.text }}'         # Shell will see the value

Context & Scope

Available Contexts

ContextAvailabilityType
inputsAll jobsInput values from trigger
varsAll jobsStatic workflow variables
needs.<job>Dependent jobsPrevious job results
matrix.<key>Matrix jobsCurrent matrix value
dynamic.<key>Dynamic jobsCurrent dynamic value
triggerAll jobsTrigger metadata
xemaAll jobsRun-level system context (org, project, run, workflow, job, actor)
resultIn outputsCurrent job result

xema.* — system-managed run context

Stable, system-populated fields available in every expression. Reach for xema.* when you want canonical run-level state instead of digging into trigger metadata.

FieldTypeDescription
xema.org.idstringOwning org id
xema.project.idstringOwning project id
xema.run.idstringWorkflow run id (stable for retries)
xema.run.attemptnumberRun attempt counter
xema.run.startedAtIsostringISO-8601 timestamp at trigger dispatch
xema.run.correlationIdstringCross-service correlation id
xema.run.actor.subjectstring | nullUser id (null for scheduled / webhook)
xema.run.actor.kindstring'user' | 'system' | 'webhook' | 'schedule'
xema.workflow.keystringWorkflow slug
xema.workflow.versionstringResolved semver
xema.job.keystringCurrent job key
xema.job.attemptnumberJob attempt counter
yaml
1# Concurrency keyed off canonical project id
2concurrency:
3  group: product-development:${{ xema.project.id }}
4  mode: queue
5
6# Job that branches on actor kind
7jobs:
8  audit:
9    if: xema.run.actor.kind == 'user'
10    uses: ...

Context Examples

yaml
1# In any job
2with:
3  # Inputs from trigger
4  project: ${{ inputs.project_name }}
5
6  # Workflow variables
7  org: ${{ vars.organization }}
8
9  # Dependent job outputs
10  previous_result: ${{ needs.analyze.outputs.summary }}
11
12  # Matrix values
13  os: ${{ matrix.os }}
14
15  # Webhook data
16  provider: ${{ trigger.provider }}
17  event: ${{ trigger.event }}

Context Hierarchy

text
1Workflow Inputs
2    ↓
3Trigger Payload
4    ↓
5Job Execution
6    ├── Access previous outputs (needs)
7    ├── Access matrix values (matrix/dynamic)
8    └── Produce new outputs (result)

Common Patterns

Pattern 1: Conditional Deployment

yaml
1jobs:
2  build:
3    uses: xema/agent
4    outputs:
5      status: ${{ result.build_status }}
6
7  deploy-prod:
8    needs: build
9    if: ${{ inputs.environment == 'production' && needs.build.outputs.status == 'success' }}
10    uses: xema/agent
11
12  deploy-staging:
13    needs: build
14    if: ${{ inputs.environment == 'staging' }}
15    uses: xema/agent

Pattern 2: Data Transformation Chain

yaml
1jobs:
2  fetch-data:
3    uses: xema/agent
4    outputs:
5      raw_data: ${{ result.data }}
6
7  transform:
8    needs: fetch-data
9    uses: xema/agent
10    with:
11      input: ${{ needs.fetch-data.outputs.raw_data }}
12    outputs:
13      transformed: ${{ result.output }}
14
15  validate:
16    needs: transform
17    uses: xema/agent
18    with:
19      data: ${{ needs.transform.outputs.transformed }}

Pattern 3: Conditional Matrix Expansion

yaml
1jobs:
2  determine-configs:
3    uses: xema/agent
4    outputs:
5      configs: ${{ result.config_list }}
6
7  build-each:
8    needs: determine-configs
9    strategy:
10      dynamic:
11        from: ${{ needs.determine-configs.outputs.configs }}
12        as: config
13    if: ${{ inputs.build_all == true }}
14    uses: xema/agent
15    with:
16      config: ${{ dynamic.config }}

Pattern 4: Aggregating Results

yaml
1jobs:
2  process-each:
3    strategy:
4      matrix:
5        item: [a, b, c]
6    uses: xema/agent
7    outputs:
8      result: ${{ result.output }}
9
10  aggregate:
11    needs: process-each
12    uses: xema/agent
13    with:
14      # Get all results from matrix jobs
15      all_results: ${{ needs.process-each.outputs[*] }}

Type Safety

Type Validation

Input types are validated before execution:

yaml
1on:
2  workflow_dispatch:
3    inputs:
4      count:
5        type: integer
6        required: true
7
8jobs:
9  use-count:
10    with:
11      count: ${{ inputs.count }}  # Always an integer

Type Conversion

Explicit conversion in expressions:

yaml
1${{ toNumber(inputs.string_number) }}
2${{ toString(inputs.number) }}
3${{ toJSON(inputs.object) }}
4${{ fromJSON(inputs.json_string) }}     # parse a JSON string at runtime
5${{ fromJSON(needs.draft.outputs.payload) }}   # parse an ArtifactRef's payload

See the Payload Reach-In section for how fromJSON interacts with deliverable specs and when to prefer typed reach-in over the escape hatch.

Null Safety

Handle missing values:

yaml
1${{ inputs.optional_field || 'default-value' }}
2${{ needs.previous.outputs.field || null }}

Examples

Example 1: Multi-Phase Data Flow

yaml
1vars:
2  max_scope: enterprise
3
4on:
5  workflow_dispatch:
6    inputs:
7      project_scope:
8        type: string
9        required: true
10
11jobs:
12  requirements:
13    uses: xema/agent
14    with:
15      scope: ${{ inputs.project_scope }}
16      template: ${{ vars.max_scope }}
17    outputs:
18      spec_id: ${{ result.artifact_id }}
19      effort_days: ${{ result.effort_days }}
20
21  architecture:
22    needs: requirements
23    uses: xema/agent
24    with:
25      requirements_id: ${{ needs.requirements.outputs.spec_id }}
26      complexity: ${{ inputs.project_scope }}
27    outputs:
28      design_id: ${{ result.artifact_id }}
29
30  estimate:
31    needs: [requirements, architecture]
32    uses: xema/agent
33    with:
34      requirements_effort: ${{ needs.requirements.outputs.effort_days }}
35      design_id: ${{ needs.architecture.outputs.design_id }}

Example 2: Webhook Trigger with Expressions

yaml
1on:
2  webhook:
3    - event: scm.pull_request
4
5jobs:
6  analyze:
7    if: ${{ trigger.payload.pull_request.state == 'opened' }}
8    uses: xema/agent
9    with:
10      pr_title: ${{ trigger.payload.pull_request.title }}
11      changed_files: ${{ trigger.payload.pull_request.files_changed }}
12    outputs:
13      review_status: ${{ result.status }}
14
15  post-review:
16    needs: analyze
17    if: ${{ needs.analyze.outputs.review_status == 'approved' }}
18    uses: xema/webhook
19    with:
20      url: ${{ trigger.payload.pull_request.comments_url }}
21      body:
22        status: approved

Example 3: Matrix with Conditional Output Processing

yaml
1jobs:
2  test-all:
3    strategy:
4      matrix:
5        version: [18, 20, 22]
6    uses: xema/agent
7    with:
8      node_version: ${{ matrix.version }}
9    outputs:
10      status: ${{ result.status }}
11
12  report:
13    needs: test-all
14    uses: xema/agent
15    with:
16      all_statuses: ${{ needs.test-all.outputs[*] }}
17      success_count: ${{ length(filter(needs.test-all.outputs, 'success')) }}

Next: Read Templates System for details on template customization.