Diagnostics
Purpose
Give a short, ordered diagnostic path when “something is wrong” before branching into Hosting vs Virtualization vs Manager-unavailable runbooks. This page is not an index-only stub; it is the first probe sequence.
Audience
On-call operators with admin JWT and systemd access.
Edition
SHARED commands. Hosting units and Hosting APIs are Enterprise.
Status
SUPPORTED as verified commands/endpoints. Log paths differ by unit.
Prerequisites
Shell on Manager and Agent hosts. zwallctl with --manager-url set to the real listener (example https://manager.example.com:8443, not unconfigured http://localhost:8080).
Required permissions
Root for systemctl/journalctl. Admin for API checks.
Procedure
systemctl status zwall-manager
systemctl status zwall-agent
systemctl status zwall-hosting-ebpf-loader
systemctl status zwall-hosting-agent
# or zwall-hosting-agent-v2
curl -k https://manager.example.com:8443/health/live
curl -k https://manager.example.com:8443/health/ready
zwallctl login --manager-url https://manager.example.com:8443 --username admin
zwallctl ping
zwallctl list-agents
zwallctl checkThen:
GET /api/v1/system/sanity-checkGET /api/v1/dashboard/ebpf-health(Enterprise; Virtualization attach)- Hosting:
GET /api/v1/agents/:id/hosting/statusand/hosting/events - Virtualization:
GET /api/v1/virtualization/vmsand/interfaces
Logs:
Scroll sideways to see the full table. Use the arrow keys when focused.
| Source | Path |
|---|---|
| Manager example YAML | /var/log/zcs/firewall-manager.log |
| Enterprise Manager unit | /var/log/zwall-manager |
| Virtualization Agent example | /var/log/zcs/firewall-agent.log |
| Hosting Agent example | stdout / empty log_file — use journal |
| Always | journalctl -u zwall-manager -u zwall-agent -u zwall-hosting-agent -u zwall-hosting-ebpf-loader |
Levels: Manager logging.level, Agent agent.log_level (debug|info|warn|error).
zwallctl stats reads local eBPF maps on a Virtualization host (pin dir /sys/fs/bpf/fw), not the Manager. Use it only on an enforcement node.
Verification
Live alive, ready ready with database: ok, Agents listed, sanity-check reachable. Then interpret empty Hosting/virt widgets using profile-specific docs — do not conclude “observability is down” from one empty panel.
Expected result
A split: Manager process vs database vs Agent enrollment vs collector ACL vs eBPF attach.
Failure modes
Scroll sideways to see the full table. Use the arrow keys when focused.
| Probe split | Next document |
|---|---|
| live fail | Manager unavailable |
| live ok, ready fail | database/TLS/listener — same |
| Manager ok, Agent missing | Agent disconnected, Enrollment |
| Agent online, no events | Missing telemetry |
| Virt attach fail | eBPF, Enforcement |
| Hosting collectors blocked | Hosting troubleshooting |
| Database | Database |
zwallctl start/stop not implemented. zwallctl log --follow not implemented.
Troubleshooting
Do not enable allow_insecure_remote_tls or allow_unsigned_artifacts as a diagnostic default. Do not load firewall.o on a Hosting node to “see counters.”
Security considerations
Example log files may contain operational data. Do not ship JWTs, database passwords, or AI keys. Redaction exists in Manager logging code; still treat journals as sensitive.
Limitations
This page does not certify a unified log directory or lossless event delivery. Health probes are unauthenticated; /metrics is not.