What database incidents AI agents can handle, and what humans must decide

この文書の適用条件

そのまま実行できるコマンド

-- 対象: SQL Server 2012 以降 / Amazon RDS for SQL Server
-- 権限: VIEW SERVER STATE
-- 変更作業: なし（参照のみ）
-- Production 実行: 可能
SELECT
    r.session_id,
    r.blocking_session_id,          -- 0 以外なら、この値のセッションに待たされている
    r.status,
    r.command,
    r.wait_type,
    r.wait_time                       AS wait_time_ms,
    r.wait_resource,
    DB_NAME(r.database_id)            AS database_name,
    s.login_name,
    s.host_name,
    s.program_name,
    r.total_elapsed_time              AS elapsed_ms,
    SUBSTRING(t.text, 1, 500)         AS sql_text
FROM sys.dm_exec_requests AS r
INNER JOIN sys.dm_exec_sessions AS s
        ON r.session_id = s.session_id
OUTER APPLY sys.dm_exec_sql_text(r.sql_handle) AS t
WHERE r.blocking_session_id <> 0
   OR r.session_id IN (
        SELECT blocking_session_id
        FROM sys.dm_exec_requests
        WHERE blocking_session_id <> 0
      )
ORDER BY r.wait_time DESC;

-- 対象: MySQL 5.7 / 8.0 系、Aurora MySQL 2 / 3
-- 権限: PROCESS 権限
-- 変更作業: なし（参照のみ）
-- Production 実行: 可能
SELECT
    trx.trx_id,
    trx.trx_state,
    trx.trx_started,
    TIMESTAMPDIFF(SECOND, trx.trx_started, NOW()) AS trx_age_sec,
    trx.trx_mysql_thread_id                       AS thread_id,
    trx.trx_rows_locked,
    trx.trx_rows_modified,                        -- 大きいほどロールバックに時間がかかる
    p.user,
    p.host,
    p.db,
    p.command,
    p.time                                        AS thread_time_sec,
    p.state,
    LEFT(COALESCE(trx.trx_query, p.info), 500)    AS current_sql
FROM information_schema.INNODB_TRX AS trx
LEFT JOIN information_schema.PROCESSLIST AS p
       ON p.ID = trx.trx_mysql_thread_id
ORDER BY trx.trx_started ASC;

-- 補助: トランザクションを持たない接続も含めて全体を見る
SHOW FULL PROCESSLIST;

-- 対象: SQL Server 2012 以降 / MySQL 5.7 以降・Aurora MySQL
-- 権限: SQL Server は ALTER ANY CONNECTION、MySQL は CONNECTION_ADMIN 相当
-- 変更作業: あり（セッションの強制終了とトランザクションのロールバック）
-- Production 実行: 人間の承認を得た場合のみ

-- 実行前に、以下の6項目すべてを確認する。1つでも未確認なら実行しない。
--   1. 対象がブロッキング連鎖の起点か（他に待たされていないセッションか）
--   2. そのセッションが何をしているか（定常のアプリ処理・バッチ・DDL のいずれか）
--   3. ロールバックの想定時間（更新行数が多いほど長く、その間ロックは解放されない）
--   4. 呼び出し元が再試行するか。しない場合、業務データが欠落しないか
--   5. 止める代わりに待つ選択肢が取れないか（業務側の許容時間の確認）
--   6. 承認者は誰か。実行の記録をどこに残すか

-- SQL Server: セッションIDを指定（値はサンプル）
KILL 57;

-- ロールバックの進捗を確認する（KILL 後に実行する参照専用クエリ）
-- KILL 57 WITH STATUSONLY;

-- MySQL / Aurora MySQL: PROCESSLIST の ID を指定（値はサンプル）
-- KILL 12345;

結果の読み方

こういう状況で使います

• Application timeouts have spiked, and it isn't yet clear whether the database is the cause
• Updates to one specific table are stuck waiting
• An incident that happened overnight went unnoticed until it was discovered the next morning
• Monitoring is in place, but there are too many alerts to tell which one is real
• Every incident, the responder has to recall which queries to run from memory
• We want AI to handle initial response, but we haven't drawn the line on how far to let it go

考えられる原因（可能性の高い順）

1. 01

   Detection exists, but triage depends on what responders remember

   If the queries to check aren't organized into a procedure, the facts gathered vary by responder. This is the part of the process that's easiest to automate.

2. 02

   Whether initial response is allowed was never decided in advance

   Without a clear answer to "which actions are safe to run automatically," teams end up either pushing everything back to manual (to stay safe) or, conversely, automating actions that are genuinely dangerous.

3. 03

   Irreversible actions are treated the same as reversible ones

   Killing a session, deleting data, and reinitializing replication are all called "responses," but they differ wildly in how easy they are to undo. Treating them the same leads to incidents.

4. 04

   No one has the authority to decide on a shutdown

   Whether to stop the service to prioritize recovery, or keep running slow, is a business call. Without a designated decision-maker, technical staff end up shouldering that decision themselves, which delays response.

確認手順

1. 1

   Check the blocking chain

   参照のみ

   Use the read-only query above to list waiting and blocking sessions, and identify the head of the chain.

2. 2

   Check active sessions and their origin

   参照のみ

   Look for concentration in a particular application or host. If found, check that application's recent changes.

3. 3

   Tally wait events

   参照のみ

   Determine whether lock waits, I/O waits, or CPU waits dominate — this determines where to look next.

4. 4

   Check transaction log usage

   参照のみ

   If the log runs out of free space, all write activity halts. Check whether a long-running transaction is preventing log reuse.

5. 5

   Check replication lag

   参照のみ

   If read traffic is routed to a replica, lag causes business impact. Check the size of the lag and whether it's increasing.

6. 6

   Check recent changes

   参照のみ

   Check whether a deployment, new batch job, or parameter change happened just before the incident. A matching timeline is a strong clue.

対応方法

バージョン・環境による違い

これで解決しない場合に確認すること

• Check how long recent incidents took from detection to completed triage

  If most of that time was spent "remembering what to check," triage automation will help.

• Classify the actions taken during initial response as reversible or irreversible

  If an irreversible action was run without approval, that's your next incident waiting to happen.

• Confirm who has authority over the shutdown decision

  Check whether it's also decided what happens during hours when that person is unavailable.

• Check your alerts' accuracy

  Count what fraction of recent alerts actually required a response. If it's low, real anomalies get buried.

• Check whether postmortem records actually get referenced during the next incident

  If they aren't being read, reconsider the format or where they're stored.

この文書の根拠と限界

よくある質問

この文書がカバーする質問

• What should human experts be responsible for in AI operations
• Is it safe to let an AI agent handle initial database incident response
• What queries should I run first during a database incident

リスク表示の意味

• 参照のみデータと設定を変更しません。
• 低影響は限定的ですが、権限と負荷の確認が必要です。
• 中性能・ロック・コストに影響する可能性があります。
• 高障害・データ損失・復旧作業が発生する可能性があります。
• 専門家レビュー必須本番適用前に別途レビューが必須です。

GIIPの対応範囲

The phase breakdown and the lines drawn above can be implemented in-house without any specific product. At GIIP, AI agents and human experts continuously monitor and operate multiple databases and roughly 30 web services across AWS and Azure; AI agents handle detection and triage, while the irreversible operations and shutdown decisions listed above are executed only after a human operator approves them. If it's hard to judge which actions count as reversible when drawing your own line, that can be worked out from your current setup and past incident records.

関連サービス

• 障害対応・火消し支援
• 24時間監視の対象を確認する
• AIインフラ運用