Workers

The Workers page shows the current runtime state of ASP's five background Workers. It helps identify cases where a container is still running but its Worker polling loop has failed.

Entry and Permissions

Workers is located in the Workers Tab of System Settings and is available only to admins.

The page loads immediately when opened and refreshes automatically every 10 seconds. It has no manual refresh button; failed requests are retried during the next refresh cycle.

Worker Types

Worker	Django Command	Purpose
Agentic Module Worker	run_agentic_module_worker	Consume Module Redis Streams and execute custom Modules.
Case Analysis Worker	run_agentic_case_analysis_worker	Execute pending Case Analysis Jobs.
Playbook Worker	run_agentic_playbook_worker	Execute pending Playbook Runs.
ELK Action Worker	run_elk_action_worker	Poll the ELK Action Index and write to Redis Streams.
Dashboard Cache Worker	run_dashboard_cache_worker	Generate Dashboard caches periodically.

State

State	Description
Starting	The Worker started but has not completed its first polling iteration.
Idle	The latest polling iteration succeeded and no iteration is currently running.
Running	The Worker is currently executing an iteration.
Degraded	The latest Worker polling iteration encountered an infrastructure error.
Down	The Worker never reported, exited normally, or its heartbeat expired.

The reason field distinguishes the causes of Down:

Reason	Description
never_reported	The Worker has never written health state.
heartbeat_expired	The latest heartbeat is more than 30 seconds old.
graceful	The Worker exited normally.

An individual Playbook, Case Analysis, or Module business failure does not mark its Worker as Degraded. Inspect the corresponding business record and logs for those failures.

Page Fields

• Current State and reason.
• Current or latest task summary.
• Current or latest iteration duration.
• Latest heartbeat time.
• Latest successful polling time.
• Latest time that work was processed.
• Latest polling failure time.
• A safe error summary without raw exceptions or secrets.

The page does not show backlogs, historical trends, log content, or cumulative counters, and provides no Restart, Stop, or Run Now actions.

Manual Execution

Compose already runs one long-lived process for each Worker type. To execute one iteration while troubleshooting, use --once:

docker compose exec asp-web python manage.py run_dashboard_cache_worker --once

Replace the command with another command from the table as needed. --once does not report Worker Health.

Do not manually start a second long-lived Worker without --once. The current Compose deployment supports only one instance of each type, and multiple instances overwrite each other's health state.

Troubleshooting

Monitoring unavailable

The Workers API reads health state from Redis. When Redis is unavailable, the page displays:

Worker health monitoring is unavailable.

Check Redis and ASP service status:

docker compose ps
./scripts/doctor.sh

Degraded or Down

Inspect the corresponding Compose service logs, for example:

docker compose logs --tail=100 asp-worker-playbook
docker compose logs -f asp-worker-dashboard-cache

After fixing the configuration or runtime problem, restart the affected service. The next automatic page refresh shows the new state after polling resumes.

Boundaries

• The heartbeat thread reports every 10 seconds.
• A heartbeat older than 30 seconds is calculated as Down.
• The platform does not automatically detect Stalled; long tasks remain Running and show their duration.
• Worker Health represents process and polling-loop state. It does not replace business results, container status, or logs.

Next Steps

• Restart & Operations — Check services, inspect logs, and restart Workers.
• Runtime — Configure runtime parameters such as the Dashboard Worker refresh interval.
• Custom Console — Inspect Module and Playbook definitions.