Why AI automated execution needs approval and rollback, and how to design for it

この文書の適用条件

そのまま実行できるコマンド

# 対象: 自組織の運用自動化ワークフロー定義
# 権限: ワークフロー定義の編集権限
# 変更作業: なし（定義の記述例）
# Production 実行: 該当なし（設計サンプル）

workflow: sample-log-maintenance
  # 実行対象を明示的に限定する。ここに書かれていないホスト・DBには一切触れない
  scope:
    allowlist_hosts: [LEGACY-SQL01]
    allowlist_databases: [SampleDB]
    deny_if_role: [primary-replica-source]

  # 実行前に必ず状態を保存する。保存に失敗したら以降のステップへ進まない
  pre_snapshot:
    - id: capture-config
      command: dump-configuration
      required: true          # 失敗時は abort（スキップ不可）
      retention: 30d

  steps:
    - id: check-log-usage
      kind: read
      dry_run: false          # 参照のみなので dry-run の概念が無い
      requires_approval: false
      idempotent: true
      rollback: not-required   # 状態を変えないため

    - id: apply-parameter-change
      kind: write
      dry_run: true            # まず差分だけを出力して人が読む
      requires_approval: true  # dry-run の出力を見た人が承認して初めて本実行
      approvers: [ops-oncall, dba-lead]
      approval_expires_in: 2h  # 古い承認で実行されないように失効させる
      idempotent: true         # 同じ入力で2回流しても結果が変わらない
      rollback:
        exists: true           # 存在しない場合、この step は定義エラーとして拒否する
        command: restore-configuration --from capture-config
        verified_at: 2026-08-13

    - id: never-auto
      kind: write
      requires_approval: true
      auto_execute: forbidden  # 承認があっても自動実行経路からは呼ばない

  # 段階的展開: 1台 → 一部 → 全体。各段で停止条件を評価する
  rollout:
    stages: [canary(1), partial(25%), full]
    halt_on: [error_rate_increase, unexpected_diff, snapshot_missing]

  # 停止スイッチ: これを立てると進行中の実行も次のステップに進まない
  kill_switch:
    flag: /etc/ai-ops/HALT
    honored_between_steps: true

  # 状態記録: 各ステップの結果を永続化し、再開時に完了済みを再適用しない
  state_store:
    key: run_id + step_id
    values: [pending, running, succeeded, failed, rolled_back, skipped]
    resume_policy: skip-succeeded

-- 対象: SQL Server 2012 以降 / Amazon RDS for SQL Server
-- 権限: 対象インスタンスへの接続権限（sys.configurations は既定で参照可能）
-- 変更作業: なし（参照のみ）
-- Production 実行: 可能
SELECT
    SERVERPROPERTY('MachineName')  AS machine_name,
    SYSDATETIMEOFFSET()            AS captured_at,
    c.configuration_id,
    c.name,
    c.value,                       -- 設定された値
    c.value_in_use,                -- 実際に効いている値
    c.is_dynamic,                  -- 再起動なしで反映されるか
    c.is_advanced
FROM sys.configurations AS c
ORDER BY c.name;

# 対象: 自組織の運用ホスト（対象DBへ接続できる場所）
# 権限: 対象DBへの接続権限と、保存先ディレクトリへの書き込み権限
# 変更作業: なし（DBの状態は変更しない。ファイルのみ出力）
# Production 実行: 可能

SNAP_DIR=/var/lib/ai-ops/snapshots
RUN_ID=sample-run-0001
mkdir -p "$SNAP_DIR/$RUN_ID"

# 1) 変更前の構成を保存する（保存に失敗したら以降へ進まない）
sqlcmd -S LEGACY-SQL01 -U sample_user -d SampleDB -i capture_configuration.sql -o "$SNAP_DIR/$RUN_ID/before.txt" || exit 1

# 2) 変更を適用する（承認済みの場合のみ。ここでは実行しない）
echo "apply step は承認後に別経路で実行する"

# 3) 変更後に同じクエリを流し、差分を人が読める形で残す
sqlcmd -S LEGACY-SQL01 -U sample_user -d SampleDB -i capture_configuration.sql -o "$SNAP_DIR/$RUN_ID/after.txt"
diff -u "$SNAP_DIR/$RUN_ID/before.txt" "$SNAP_DIR/$RUN_ID/after.txt" > "$SNAP_DIR/$RUN_ID/diff.txt"

# 4) 差分が空でないこと（=意図した変更が入ったこと）と、想定外の行が無いことを確認する
cat "$SNAP_DIR/$RUN_ID/diff.txt"

結果の読み方

こういう状況で使います

• An automated process was supposed to be running, but no one can tell since when it stopped
• A production setting changed at some point, and no one can identify who changed it
• Retrying the same automated run caused the change to be applied twice
• You want to stop an automated run, but the only way is to kill the process
• Automated-run logs exist, but the approver and input values aren't recorded
• There's still a path where the real run executes without anyone looking at the dry-run output first

考えられる原因（可能性の高い順）

1. 01

   Detection and execution are wired into the same path

   When an agent that detects an anomaly also executes the response itself, a detection error turns directly into a state change. Inserting approval between detection and execution keeps a false positive from changing state.

2. 02

   The execution target is resolved dynamically from instructions

   Resolving a target host or database name from natural-language instructions can match an unintended target. Fixing the target statically via an allowlist closes this path.

3. 03

   Retries aren't idempotent

   A retry after a timeout, or a resume after a partial failure, can reapply a change that already succeeded. Without per-step state tracking, there's no way to tell how far execution got.

4. 04

   Rollback is treated as something to 'figure out later'

   An operation with no way back at execution time loses its options the moment it fails. Making the existence of a rollback procedure a precondition for execution structurally prevents this.

5. 05

   The application log is used as a substitute for an audit log

   Application logs record what ran, but not who approved it or when. Without being able to reconstruct 'whose decision was this' after the fact, root-cause analysis becomes guesswork.

確認手順

1. 1

   List all automated runs

   参照のみ

   Enumerate the automated processes currently running and classify each as state-changing or read-only. Anything that can't be classified should be demoted to read-only for now.

2. 2

   Check whether each automated run has a rollback procedure

   参照のみ

   For anything that changes state, check whether a way back is documented and whether it has ever actually been tried.

3. 3

   Check that all five audit-log fields are present

   参照のみ

   Pick one recent automated run and try to reconstruct the executor, execution time, input, output, and approver.

4. 4

   Verify the kill switch in a test environment

   中

   Raise the halt flag and confirm an in-progress run doesn't proceed to the next step. Do not do this in production.

5. 5

   Verify double-application on retry in a test environment

   中

   Deliberately fail partway through, then resume, and confirm steps that already succeeded are not re-run.

対応方法

バージョン・環境による違い

これで解決しない場合に確認すること

• Pick one recent automated run and try to reconstruct the 5W1H

  Test whether you can reconstruct who ran it, when, what, with what input, and under what approval — using only the logs.

• Check that the approver isn't the same person as the executor

  A setup where only the same person can approve is an approval gate in name only.

• Check who can trigger the kill switch

  Confirm whether the on-call person can stop it alone if something goes wrong in the middle of the night.

• Check that the snapshot storage doesn't share a fate with the target

  Storing it on the same server means the snapshot is lost along with that server if it's lost.

• Decide when to review the allowlist

  If the allowlist isn't updated as targets are added or removed, it drifts from reality.

この文書の根拠と限界

よくある質問

この文書がカバーする質問

• How should sequential execution and state tracking be designed for AI workflows?
• How do you build an approval flow when delegating production operations to AI?
• I want a list of database operations that must not be automated

リスク表示の意味

• 参照のみデータと設定を変更しません。
• 低影響は限定的ですが、権限と負荷の確認が必要です。
• 中性能・ロック・コストに影響する可能性があります。
• 高障害・データ損失・復旧作業が発生する可能性があります。
• 専門家レビュー必須本番適用前に別途レビューが必須です。

GIIPの対応範囲

Everything above can be implemented without any specific orchestrator product. At GIIP, AI agents and human experts continuously monitor and operate multiple databases and roughly 30 web services across AWS and Azure; AI agents are allowed to execute only operations whose impact is reversible and whose targets are limited, and the operation classes listed above go through human approval. If you're not sure where to draw this line for your own organization, it's also possible to work it out starting from your current list of automated processes.

関連サービス

• AIインフラ運用
• GIIP FDE Ops の対応範囲
• AIエージェントによるIT運用