SDK — Testing

A biome is testable at four levels, from cheapest to most expensive:

  1. Manifest validation — parse xema-biome.json against the kernel schema.
  2. Contribution unit tests — each contribution (agent, workflow, handler) is testable in isolation.
  3. Capability-stub tests — mock the gateway, assert that the biome calls the right capabilities with the right inputs.
  4. End-to-end — install into a sandbox, exercise the biome through real capability calls.

This page covers the first three. End-to-end tests share infrastructure with the platform's own dev test-suite at test-suite/.

1. Manifest validation

The kernel schema (BiomeManifestSchema in @xemahq/contracts/biome) is the single source of truth. Validate at unit-test time with the same Zod parser the host uses at install:

ts
1// tests/manifest.spec.ts
2import { describe, it, expect } from 'vitest';
3import { readFileSync } from 'node:fs';
4import { BiomeManifestSchema } from '@xemahq/contracts/biome';
5
6describe('xema-biome.json', () => {
7  it('parses against the kernel schema', () => {
8    const raw = JSON.parse(readFileSync('./xema-biome.json', 'utf8'));
9    expect(() => BiomeManifestSchema.parse(raw)).not.toThrow();
10  });
11});

The parser refuses unknown closed-set values fail-fast. Run this test in your biome's CI before any other step — a manifest that fails parse cannot install.

2. Contribution unit tests

Each contribution is a pure(-ish) module: an agent definition is a YAML document, a workflow definition is YAML, a handler is a TS function. Test them the way you would test any other unit.

Agent definitions — the fastest and most complete check is xema biome validate, which runs the platform's own pre-boot pass over the whole biome directory: manifest schema, xema.agents[] ⟷ agents/*.md parity, contribution envelopes, skill frontmatter and workflow schemas. It exits non-zero listing every issue, so it drops straight into CI:

bash
1xema biome validate

For assertions on a specific agent, the agent contracts live in @xemahq/contracts/agent-composition (Agent, AgentRef, AgentLifecycle, AgentLimits, ResolvedAgent).

Workflow definitions — parse with the DSL compiler from @xemahq/dsl. The compiler runs the same closed-set checks the platform runs at install time.

Event handlers — import the handler module, build a typed event envelope by hand, pass a mock context, assert on the calls the handler makes (see below).

3. Capability-stub tests

Biome runtime code never calls platform services directly — it calls capabilities through ctx.callCapability(...). To test what your handler does on a given event, replace callCapability with a spy:

ts
1import { describe, it, expect, vi } from 'vitest';
2import onRunCompleted from '../dist/handlers/on-run-completed.js';
3
4describe('on-run-completed', () => {
5  it('posts a chat message when the run succeeds', async () => {
6    const callCapability = vi.fn().mockResolvedValue({ ok: true });
7
8    await onRunCompleted(
9      {
10        type: 'workflow.run.completed.v1',
11        source: 'workflow-engine-api',
12        id: 'evt-1',
13        time: '2026-05-26T12:00:00Z',
14        subject: 'xema://orgs/acme/projects/main/workflow-run/r-1',
15        orgid: 'org_acme',
16        projectid: 'proj_main',
17        data: { workflowRunId: 'r-1', outcome: 'success' },
18      },
19      { callCapability, log: console, installationId: 'inst-1' } as any,
20    );
21
22    expect(callCapability).toHaveBeenCalledTimes(1);
23    expect(callCapability).toHaveBeenCalledWith(
24      'connector:channel.send-message@1',
25      expect.objectContaining({ channel: 'team-engineering' }),
26    );
27  });
28
29  it('is idempotent on retries', async () => {
30    const callCapability = vi.fn().mockResolvedValue({ ok: true });
31    const event = makeEvent({ id: 'evt-1' }); // same id
32
33    await onRunCompleted(event, { callCapability } as any);
34    await onRunCompleted(event, { callCapability } as any);
35
36    // Handler must dedupe internally on event.id
37    expect(callCapability).toHaveBeenCalledTimes(1);
38  });
39});

The pattern generalises:

  • What you assert — which capability refs the biome calls, with which inputs, in which order, and that it does not call refs outside its declared requiresCapabilities[].
  • What you do not assert — what the platform does after the capability call. That is the platform's contract, not the biome's.

Asserting the manifest matches the runtime

A common bug is a handler that calls a capability the manifest does not declare. Catch it in CI:

ts
1import { readFileSync } from 'node:fs';
2import { BiomeManifestSchema } from '@xemahq/contracts/biome';
3
4it('every capability the handler calls is in requiresCapabilities[]', () => {
5  const manifest = BiomeManifestSchema.parse(
6    JSON.parse(readFileSync('./xema-biome.json', 'utf8')),
7  );
8  const declared = new Set(manifest.xema.requiresCapabilities ?? []);
9
10  // calls is the set asserted in your handler unit tests
11  const calls = new Set([
12    'connector:channel.send-message@1',
13    'biome-storage:collection.write@1',
14  ]);
15
16  for (const ref of calls) {
17    expect(declared.has(ref)).toBe(true);
18  }
19});

Running this as a unit test means a misdeclared manifest fails before publish — the alternative is a runtime denial at install time.

Storage tests

No standalone collection-schema parser ships today. There is no standalone storage SDK package and no exported CollectionSchema validator to call from a unit test. Validate declared collections through xema biome validate, which runs the platform's own manifest pass over the biome directory.

Lifecycle hook tests

There is nothing to test. The manifest's lifecycle block is declared but never invoked — no host resolves the module paths it names, so a hook module has no caller to test against. See Lifecycle Hooks.

Install-time work that actually runs belongs in your biome service's startup path; test it the same way you test any other service startup.

End-to-end against a sandbox

The cheapest path to a real end-to-end run is to install the biome into a sandbox environment via the existing platform install flow (today: POST /biomes/install; Phase 5+: xema biome install ./your-biome --environment sandbox). Sandbox runs have no production credentials and no production data; the gateway denies any call that asks for them.

The platform's own test-suite (test-suite/) gives a working local stack. See the test-suite README for the docker-compose stand-up.

Previous: ← Events I subscribe