Use Case: Multi-Repo Audit with Custom Overlays

Complexity: Advanced
Trigger: Manual dispatch or weekly schedule
Deliverables: Per-repo audit report, consolidated findings, team-specific deliverables shaped by overlays


Goal

Run a consistent audit across many repositories while allowing each team to customise the output format, depth, and criteria through deliverable spec overlays. A shared base spec defines the common audit structure; overlays add team-specific requirements (e.g. frontend teams get accessibility checks, backend teams get OWASP checks, infra teams get IaC policy checks).

This demonstrates Xema's overlay system at scale — the most sophisticated use of the templates and deliverable specs feature.


Trigger

Weekly cron schedule (Monday 06:00 UTC) or manual dispatch with an optional targetTeam filter.


Architecture

text
1Base Spec: repo-audit-v1
2    │
3    ├── overlays/frontend-audit   → adds accessibility + bundle-size criteria
4    ├── overlays/backend-audit    → adds OWASP Top 10 + API contract checks
5    └── overlays/infra-audit      → adds IaC policy + secret-scanning criteria
6
7For each repo:
8  1. Resolve effective spec = base spec + team overlay
9  2. Run agent audit against effective spec
10  3. Emit per-repo artifact
11
12Aggregate:
13  1. Merge per-repo artifacts
14  2. Produce consolidated findings report
15  3. Post Slack digest

Steps

StepTypeDescription
resolve-reposAgent actionFetches the list of repositories from the project registry, grouped by team
audit-repo (matrix)Agent action (parallel)Audits each repository using its team-specific overlay
aggregateAgent actionMerges all per-repo results into a consolidated report
post-digestHTTP actionPosts a Slack digest to each team channel
open-tickets (conditional)Agent actionCreates Jira issues for critical findings per team

Deliverables

  • Per-repo repo-audit-<slug> — Individual audit report shaped by the team's overlay. Stored as a versioned artifact.
  • consolidated-audit-report — Organisation-wide view. Top-level findings, team summaries, trend data.
  • Jira issues — One per critical finding per team, assigned to the team lead.

Workflow YAML

yaml
1apiVersion: workflow/v1alpha1
2kind: Workflow
3metadata:
4  name: multi-repo-audit-overlays
5  version: "1.0.0"
6
7on:
8  schedule:
9    cron: "0 6 * * 1"
10    timezone: UTC
11  manual:
12    inputs:
13      - name: targetTeam
14        type: string
15        required: false
16        description: "Filter to a specific team (frontend, backend, infra). Omit for all teams."
17
18inputs:
19  targetTeam:
20    type: string
21    source: "{{ trigger.inputs.targetTeam | default: '' }}"
22
23jobs:
24  resolve-repos:
25    action: [email protected]
26    input:
27      task: |
28        Fetch all active repositories from the project registry.
29        For each repository, determine the owning team (frontend, backend, infra, other)
30        and the appropriate deliverable spec overlay.
31        Filter to team {{ inputs.targetTeam }} if specified.
32        Return a list of objects: { repo, team, overlayId }.
33      context:
34        targetTeam: "{{ inputs.targetTeam }}"
35    output:
36      artifactKey: repo-list
37
38  audit-repo:
39    needs: [resolve-repos]
40    strategy:
41      matrix:
42        entry: "{{ jobs.resolve-repos.outputs.artifact.repos }}"
43      maxConcurrency: 5
44    action: [email protected]
45    input:
46      task: |
47        Perform a comprehensive code audit of the repository {{ matrix.entry.repo }}.
48        Use the deliverable spec overlay {{ matrix.entry.overlayId }} (extends repo-audit-v1).
49        Assess: code quality, security posture, dependency health, test coverage,
50        documentation completeness, and any overlay-specific criteria.
51        Produce a structured audit report.
52      context:
53        repo: "{{ matrix.entry.repo }}"
54        team: "{{ matrix.entry.team }}"
55      deliverableSpec: repo-audit-v1
56      deliverableSpecOverlay: "{{ matrix.entry.overlayId }}"
57    output:
58      artifactKey: "audit-{{ matrix.entry.repo | slugify }}"
59
60  aggregate:
61    needs: [audit-repo]
62    action: [email protected]
63    input:
64      task: |
65        Merge all per-repository audit reports into a consolidated organisation-wide
66        audit report. Structure by team. Highlight: top critical findings, most
67        improved repos, most regressed repos, and overall health score trend.
68        Compare to previous week's report if available.
69      context:
70        auditResults: "{{ jobs.audit-repo.outputs }}"
71      deliverableSpec: consolidated-audit-report-v1
72    output:
73      artifactKey: consolidated-audit-report
74
75  post-digest:
76    needs: [aggregate]
77    strategy:
78      matrix:
79        team: [frontend, backend, infra]
80    action: biomes.slack-notifier.post-[email protected]
81    input:
82      channelId: "{{ env['TEAM_SLACK_CHANNEL_' + matrix.team | upper] }}"
83      message: |
84        *Weekly Repo Audit — {{ now | date: '%Y-%m-%d' }} — {{ matrix.team | capitalize }} Team*
85        {{ jobs.aggregate.outputs.artifact.teamSummaries[matrix.team] }}
86        Full report: {{ jobs.aggregate.outputs.artifact.url }}
87
88  open-tickets:
89    needs: [aggregate]
90    if: "{{ jobs.aggregate.outputs.criticalFindingsCount > 0 }}"
91    strategy:
92      matrix:
93        finding: "{{ jobs.aggregate.outputs.criticalFindings }}"
94    action: [email protected]
95    input:
96      task: |
97        Create a Jira issue for the critical finding below.
98        Project: OPS. Issue type: Bug. Priority: Critical.
99        Assign to the team lead of team {{ matrix.finding.team }}.
100        Include: repo, file/area, finding description, recommended remediation.
101      context:
102        finding: "{{ matrix.finding }}"

Defining Overlays

Overlays are managed through the deliverable-specs-api. An overlay extends a base spec by adding or overriding criteria sections.

Example: overlays/backend-audit overlay definition:

yaml
1apiVersion: deliverable-spec/v1alpha1
2kind: Overlay
3metadata:
4  id: backend-audit
5  extends: repo-audit-v1
6
7additions:
8  sections:
9    - id: owasp-checks
10      title: "OWASP Top 10 Assessment"
11      required: true
12      schema:
13        type: object
14        properties:
15          injectionRisks:
16            type: string
17          authenticationFlaws:
18            type: string
19          sensitiveDataExposure:
20            type: string
21          securityMisconfiguration:
22            type: string
23          overallOwaspScore:
24            type: string
25            enum: [pass, warn, fail]
26        required: [overallOwaspScore]
27
28    - id: api-contract-health
29      title: "API Contract Health"
30      required: true
31      schema:
32        type: object
33        properties:
34          openApiSpecPresent:
35            type: boolean
36          breakingChangesDetected:
37            type: boolean
38          clientGenerationPassing:
39            type: boolean

See the Overlays documentation for the full overlay DSL reference.


Extending It

  • Historical trending — Store consolidated reports in a time-series artifact store and compare week-over-week health scores.
  • Score-based gate — Block the pipeline if the organisation-wide health score drops below a threshold.
  • Per-repo drill-down sessions — For repos with critical findings, automatically launch an interactive engineering session to remediate.
  • External report export — Add a step that renders the consolidated report as a PDF and emails it to engineering leadership.
  • Dynamic overlay selection — Derive the overlay ID from repo metadata (language, stack, compliance tier) rather than hardcoding team names.

Previous: ← Feature Lifecycle Pipeline
Back to index: Use Cases Overview