Service Monitor Guide

Watch the real-time status, health, and execution history of the services and cron scripts running across the GIIP ecosystem from a single admin screen. Skim the overall picture with the summary dashboard, then drill into individual services with the List, Flow, and Trigger views.

🖥️ Go to the Service Monitor page →

📋 Overview

The Service Monitor page is the admin console for observing the real-time state of the cron scripts, agent workers, and Azure Functions running across the GIIP ecosystem. It lists each service's last reported status (running / idle / error / no report), execution time, and telemetry (success, fail, skipped, etc.) in a table, and the summary cards at the top give a one-glance view of overall health. The data comes from status reports that agents push into the time-series store (tKVS); this screen is where an operator decides "what is running and what has stopped."

⚠️ This screen is admin-only. Access requires admin level uLevel >= 70. Users below that level are redirected to home.

🔍 Screen Layout

1. Top Header

• Page title/description: "Service Monitor" (localized) plus a summary line.
• View toggle: switches among the List / Flow / Trigger views.
• Add Script button (List view only, giip #1307): opens a modal to register a new monitored script.
• Refresh button: immediately reloads service status (icon spins while loading). The List view auto-refreshes every 30 seconds; the Flow and Trigger views every 60 seconds.

2. Summary Dashboard

Four cards at the top show aggregate counts for all services.

Card	Meaning
Total	Total number of monitored services/scripts
Running	Count of services currently in RUNNING state
Healthy	Count of services reporting IDLE (normal standby)
Error	Count of services reporting ERROR (failed/unhealthy)

3. List View (Service Table)

Each row is one service/script; click a row to expand it and see details (sub-tasks and metrics).

Column	Description
Script Name	Script/service name
Migration	Azure Function migration status (completed / pending / none)
Hostname	Host that reported the status
Status	Status badge (RUNNING / IDLE / ERROR / UNKNOWN)
Last Execution	Last run time
Telemetry	Run metrics such as success, fail, skipped, killed
Details	Expand to view sub-tasks and the raw report

Each row has up to two action icons on the right (clearly separated as of giip #1307):

Icon	Label	Action	Applies to
🚫 (circle-X, amber)	"Remove from monitoring"	Turns off tServiceMonitorScript.enabled (soft delete). Does not touch tKVS telemetry.	Any row sourced from the monitored-script list (including UNKNOWN rows with no raw/ksn)
🗑️ (trash, red)	"Delete" (giip #1214)	Deletes one tKVS telemetry record by ksn.	Only rows with raw.ksn (i.e. a report actually arrived)

⚠️ These two buttons do completely different things. The red trash icon only removes one telemetry history entry. To make the script row itself disappear from the screen — especially a UNKNOWN row that never reported — you must use the amber "Remove from monitoring" icon instead. This closes the gap from giip #1214, where there was no way to remove UNKNOWN rows (giip #1307).

4. Flow View

Visualizes the dependencies and data flow between services (ServiceMonitorFlow) — see which service feeds into which as a flow diagram.

5. Trigger View

Shows the real-time evaluation status of registered alert triggers (tKVSTrigger) in a table. The batch SP dbo.pKVSCheckSchd re-evaluates these conditions every 5 minutes (Azure Function KVSCheckSchdTimer, giip #2098), refreshing ktChkDt, and any trigger that is over threshold with the Mail channel enabled actually gets emailed.

What each column means

Column	Meaning
Key	Identifier of the monitored target. When ktype='lssn', this is a server (lssn) number.
Factor	Name of the monitored metric (e.g. DISKUSAGE, CPUUSAGE) — the same key used when the real report lands in tKVS. If this is empty, the row can never join tKVS and is permanently "Inactive" (see the badge table below).
Condition	Shown as kAttrib kLogic kVal (e.g. Capacity gt 90 = "Capacity value exceeds 90"). kAttrib='giipLogDate' is a special case comparing "seconds since the last report" (e.g. giipLogDate gt 600 = "no report for 10+ minutes").
Last Report	When this key+factor combination last received a real value in tKVS. - means no report at all in the last 2 days.
Last Check	When the batch (pKVSCheckSchd) last evaluated this trigger (refreshes every 5 minutes). If this is stale, the batch itself has stopped — check whether KVSCheckSchdTimer (Azure Function) is running first.
Alert Channels	Which of Mail / SMS / Slack are enabled on this trigger. Mail is actually delivered (giip #2098, reusing the active SMTP config in tEmailServerConfig + giipApiSendEmail). SMS is shown but not actually sent — no delivery gateway is wired up. Slack is only sent when the trigger's own webhook (kSlackWebhook) is configured; as of 2026-09-07 zero triggers have one configured. Mail recipient decision rules (giip #2412): (1) If kMail is not empty and not '-', that value; (2) otherwise, if that usn has a registered default in tKVSTriggerDefaultMail, that default address; (3) if neither exists, no email is sent — only a [WARN] no recipient for ktSn=... log entry is written. kMailAct=0 suppresses all mail regardless of the above.
Status	One of the four badges below.

Status badges

Badge	Condition	Meaning
OK (green)	Factor present + recent report exists + condition not exceeded	Working normally. No action needed.
Overdue (red, pulsing)	Factor present + condition exceeded	A real anomaly. See "Worked examples" below.
No Data (gray)	Factor present + no report in the last 2 days	The monitored target itself isn't reporting (the agent may be down).
Inactive (gray, dimmed, hidden by default)	Factor is empty	The join to tKVS can never succeed, so this is a permanently dead trigger definition. Not a real anomaly, so it's hidden from the list by default — click "Show inactive triggers" at the top of the table to reveal it (giip #2098). To clean it up, fill in kfactor on that tKVSTrigger row in the DB, or delete the row.

Worked examples (condition combination → meaning → action)

Key	Factor	Condition	Status	Meaning	Action
584	DISKUSAGE	Capacity gt 90	Overdue (red)	Disk usage on server lssn=584 exceeded 90%	Log into that server (lssn 584) and clean up disk space (logs/temp files) or expand the volume. If the Mail channel is enabled, an email has already gone out to kMail.
2622	CPUUSAGE	giipLogDate gt 600	Overdue (red)	The CPUUSAGE report from server lssn=2622 has been missing for 600+ seconds (10+ minutes)	Check whether the agent (giipAgent) process/scheduler on that server is still alive.
(any)	(any)	(any condition)	No Data (gray)	No report for this key+factor at all in the last 2 days	Check the agent install / network / secret key first. The trigger definition itself may also be a stale typo (factor name mismatch).
(any)	(empty)	Looks incomplete	Inactive (gray, dimmed, hidden by default)	A dead definition that can never match because the factor is missing	If real monitoring is still needed, fill in kfactor and reconfigure it; otherwise delete it from the DB.

🛠️ How to Check Service Health

1. On entry, check the Error count in the summary cards first.
2. In the List view, find rows whose status badge is ERROR or UNKNOWN (no report).
3. Click the row to expand and review Last Execution / Telemetry / sub-tasks — a stale last-run time or high failure metrics flags a stopped-service candidate.
4. Use Refresh to reload the latest report and confirm whether it was just a transient delay.
5. Switch to the Trigger view to cross-check any triggers that are past their expected interval (overdue).

💡 Notes

• Status values: RUNNING = executing, IDLE = normal standby (counted under "Healthy"), ERROR = failed, UNKNOWN = no recent report (the agent has not pushed status to tKVS).
• Many UNKNOWNs usually means the agent/script is not reporting telemetry rather than that the service is dead — check whether the agent itself is running first.
• Auto refresh: the List view refreshes every 30s and Flow/Trigger every 60s, so values may change periodically.
• Access is prop mode with minLevel={70} hardcoded on the page, so uLevel >= 70 is always required (independent of menu config).

API Reference

This page talks to the backend through dispatch commands (via fetchAzureCommand). Since there is no separate API guide, the essentials are documented here.

Status polling (tKVS)

Command	Purpose	Backend SP
ApiServiceMonitorGetStatus	Fetch tKVS status (RUNNING/IDLE/ERROR) for monitored scripts (30s polling)	pApiApiServiceMonitorGetStatusbyAK
ApiServiceMonitorGetFlow	Fetch nodes/edges for the Flow view	(flow SP)
ApiServiceMonitorGetTriggerStatus	Fetch per-trigger execution status for the Trigger view	pApiApiServiceMonitorGetTriggerStatusbyAK
ApiServiceMonitorDelete	Delete one service-status record by ksn (List view trash button; a confirm modal is required). Guarded: the backend deletes only if the row's kFactor is a service-monitor factor, else returns 403. Payload { ksn }. (giip #1214)	pApiApiServiceMonitorDeletebyAK

Monitored script CRUD (giip #1307, tServiceMonitorScript)

This list decides whether a row appears on screen at all. It used to be hardcoded in the CRON_SCRIPTS array in useServiceStatus.ts; it has been migrated to this table + API so the list can be managed from the UI without a code deploy.

Command	Purpose	Backend SP
ApiServiceMonitorScriptList	List monitored scripts (enabled=1, ordered by sortOrder). Payload may be empty ({}).	pApiApiServiceMonitorScriptListbyAK
ApiServiceMonitorScriptAdd	Register a new script (the "Add Script" modal). Payload { scriptName(required), category?, azureFunction?, migrationStatus? }. If a previously removed script with the same name exists it is re-enabled (200); otherwise a new row is created (201); an already-active duplicate returns 409.	pApiApiServiceMonitorScriptAddbyAK
ApiServiceMonitorScriptDelete	Remove from monitoring (amber icon). Does not touch tKVS — only turns off tServiceMonitorScript.enabled, so UNKNOWN (never-reported) rows can be removed too. Payload { scriptId } or { scriptName }.	pApiApiServiceMonitorScriptDeletebyAK

• The response is an array of result sets (e.g. [data, status]); the client reads the first (data) array.
• A status field RstVal = 200 (List/Delete/re-enable) or 201 (new Add) means success; 409 means already monitored; 401 means the session expired or is unauthenticated (re-login required).
• Sanctuary (giipfaw run.ps1) untouched: giipApi routes command names to SP names generically, so adding SPs in giipdb alone is enough — no giipfaw code change (same pattern as giip-1214).

Troubleshooting

Symptom	Cause	Resolution
Bounced to home on load	uLevel below 70	Log in with an admin account (level 70+).
"Session expired" / 401 error	Auth token expired or missing	Log in again, then press Refresh.
Most rows show UNKNOWN	Agents/scripts are not reporting status to tKVS	Verify the agent is running and check the telemetry report path.
Many dimmed gray "Inactive" rows appear	Trigger has an empty kfactor and can never match tKVS (expected, not a real anomaly)	Hidden by default. Click "Show inactive triggers" at the top if you need to review them, then fill in kfactor or delete the row in the DB (giip #2098).
An "Overdue" (red) trigger exists but no mail arrives	kMailAct is off, both kMail and the usn's default address (tKVSTriggerDefaultMail) are empty, or the batch (KVSCheckSchdTimer) hasn't evaluated this trigger yet	Check that "Last Check" is recent (batch is running) and confirm "Alert Channels" shows Mail. If Mail is on but no mail arrives, verify kMail in the trigger row AND the usn's default address in tKVSTriggerDefaultMail in the DB — if kMail is empty but a usn default exists, it is used as the fallback (giip #2412).
"Alert Channels" shows SMS/Slack but nothing arrives	SMS has no delivery gateway wired up yet; Slack only fires when the trigger's own webhook is actually configured (0 triggers have one as of 2026-09-07)	Only the Mail channel is actually delivered today. File a giip issue if SMS/Slack integration is needed.
Values keep changing	30s/60s auto-refresh is running	This is expected. Capture the values if you need a fixed snapshot.
Top guide button (📖) not visible	Guide mapping not deployed (older build)	It appears once this guide is deployed and indexed.
UNKNOWN row won't go away	You clicked the red trash icon, but that only deletes a tKVS record — it never shows on UNKNOWN rows (no ksn) in the first place	Use the amber "Remove from monitoring" icon instead (giip #1307).
Script doesn't appear after "Add Script"	Same name is already registered and active, so Add returned 409	Check the list for an existing row with that name.

---

Version: 1.2 Last Updated: 2026-09-07 (giip #2098: Trigger view UI improvements (hide dead triggers, show real alert-channel status) + notification batch revival) Source File: giipv3/public/help/service-monitor.en.md