Continuous Monitoring & Human Oversight

Every agent step is logged: input, reasoning trace, tool calls, tool results, memory reads and writes, and final output. Logs are immutable, time-stamped, and attributable.

Per-step logging is the runtime substrate that makes everything else possible. Without it, no incident is investigable, no drift is detectable, and no control is auditable. MON-01 requires that the full trace of agent reasoning and action be captured, retained, and accessible for retrospective review. In multi-agent topologies, MAS-06 extends per-hop logging requirements.

Logs are retained per regulatory and operational requirements and stored in tamper-evident form.

Audit logs of agent actions are retained for a documented period appropriate to the regulatory environment and stored in tamper-evident form. Tamper evidence ensures that logs cannot be modified after the fact — critical for incident investigation, regulatory response, and assessment evidence. Retention periods match the requirements of applicable law and the organization's data retention policy.

Real-time anomaly detection is performed against the deployment baseline, covering volume, tool-use patterns, error rates, output distributions, and fairness metrics.

The agent's behavior in production is compared in real time against the deployment baseline captured under SPC-06. Anomaly detection covers volume, tool-use patterns, error rates, output distributions, and fairness metrics. Anomalies that cross documented thresholds trigger alerts and, for critical anomalies, automatic escalation to the human review queue.

Drift monitoring tracks quality scores, refusal rates, and performance on the standing evaluation set over time. Material drift triggers re-assessment under Domain 2.

The agent's behavior is compared against its specification over time through continuous drift monitoring. Quality scores, refusal rates, and performance on the standing evaluation set are tracked. Material drift — a sustained departure from the behavior baseline — triggers re-assessment under Domain 2 and the change-management process under GOV-05.

A human review queue receives flagged interactions, with documented SLAs for triage and resolution.

MON-05 is the operational heart of HI-AAF. The human review queue receives every escalation the agent generates, every action flagged by guardrails, and a documented sample of routine decisions for quality calibration. The queue operates with documented service-level agreements — critical items within 15 minutes, high within 4 hours, medium within 24 hours, low within 5 business days.

Reviewer qualification and training are documented. Reviewer accuracy is measured and reviewed.

Reviewers in the human review queue are qualified through documented assessment before being assigned to a queue and receive training appropriate to the agents and decisions they review. Reviewer accuracy is measured against gold-standard samples on a documented cadence, ensuring that the human layer is itself operating at a sufficient quality level.

An incident response plan specific to agent failures is in place, covering containment, investigation, customer communication, regulatory notification, and post-incident review.

Incidents involving AI agents are governed by a documented response process that covers containment, investigation, customer communication, regulatory notification, and post-incident review. The response plan is agent-specific — it accounts for the unique characteristics of agent failures, including the need to review agent reasoning traces, the possibility of cascading failures in multi-agent systems, and the challenge of determining root cause in non-deterministic systems.

Post-incident reviews feed corrective actions into the agent's evaluation set, behavior specification, and policies, closing the learning loop.

Post-incident reviews feed corrective actions back into the agent's evaluation set (SPC-07), behavior specification (SPC-01), and organizational policies — closing the learning loop. Without this feedback mechanism, the organization is condemned to repeat the same failures. Each incident should result in at least one new test case, policy update, or specification change.

Alerting and on-call coverage are defined for critical agents.

Critical agents have defined alerting rules and on-call coverage ensuring that anomalies, incidents, and system failures are surfaced to a responsible human within a documented latency. Alerting is not just monitoring — it ensures that a human who can act is notified when action is needed, at any time of day.

Monitoring effectiveness is reviewed at least quarterly: false-positive rates, missed incidents, mean time to detect, and mean time to remediate.

The monitoring system itself is reviewed at a documented cadence — at least quarterly — for effectiveness. The review covers false-positive rates, missed incidents, mean time to detect, and mean time to remediate. This meta-monitoring ensures that the monitoring controls are not just present but are actually working and improving over time.

Human reviewers are protected through exposure-limiting workflow controls for harmful content categories, documented welfare and rotation policies, conflict-of-interest screening, training on psychological safety, and a confidential channel for raising concerns.

Human reviewers operating under MON-05 are protected through exposure-limiting workflow controls for harmful content categories, documented welfare and rotation policies, conflict-of-interest screening, training on psychological safety, and a confidential channel for raising concerns about review work or content encountered. This control recognizes that the human layer has human costs.