Agentic DevOps

Three paths to agents that learn — from CVE skills to feedback loops

View the Project on GitHub jennyf19/agentic-devops

Agent Signals Protocol

Signal Types

Type Purpose Emitted by
execution Agent’s self-assessment after completing a task The working agent
outcome Independent quality evaluation A separate evaluator agent or human
escalation Agent flagging it needs human help The working agent
partnership One agent reviewing another’s signal patterns A pattern-reviewing agent

Schema

See examples/ for complete JSON examples of each signal type. The examples are illustrative; the field contracts below are normative.

Self-Assessment Scale

All self-assessment dimensions use a 1–5 integer scale:

Score Meaning
1 Poor — significant issues, low confidence
2 Below average — notable gaps or uncertainty
3 Adequate — functional but with known limitations
4 Good — solid work with minor gaps
5 Excellent — comprehensive and confident

Trust Equation

Compare the agent’s self-assessed confidence against the independent quality rating to produce a calibration score on the same 1–5 scale:

calibration = 5 - |self_assessment.confidence - quality_rating|

  5    →  perfectly calibrated (agent knows exactly what it knows)
  4    →  well-calibrated (minor difference)
  3    →  moderate miscalibration (investigate direction)
  ≤ 2  →  significant miscalibration (retrain or adjust)

Higher is better — same direction as the self-assessment scale. The gap between what an agent thinks happened and what actually happened is where the learning lives.

Common Fields

All signals must be JSON objects containing:

Signal Field Contracts

execution signal

Required fields:

Optional fields:

outcome signal

Required fields:

Optional fields:

escalation signal

Required fields:

Optional fields:

partnership signal

Required fields:

Optional fields:

Privacy

Signals must never contain:

Use bucketed values over precise ones. Prefer opt-in over opt-out.