eBPF

Kernel / eBPF / capabilities

Symptom #

• Virtualization Agent exits at start (eBPF preflight, not root, missing /sys/fs/bpf)
• XDP/TC attach failure
• Hosting network telemetry missing because the pass-only loader failed
• verify_artifact_integrity rejects objects
• operators expected UID/GID filtering to drop packets

Likely verified causes #

1. Virtualization Agent not running as root with the unit’s bounded capabilities.
2. /sys/fs/bpf not writable by the unit (ReadWritePaths includes /sys/fs/bpf).
3. Kernel below ebpf.min_kernel_version (example "5.8.0") with strict_kernel_version_check: true.
4. Wrong object: Virtualization firewall.o / firewall_egress.o vs Hosting loader hosting_observe.o (pass-only).
5. verify_artifact_integrity: true checksum mismatch; allow_unsigned_artifacts: false in the Virtualization example.
6. xdp_attach_mode (auto|native|generic|offload) incompatible with this NIC/driver.
7. Hosting loader missing ZWALL_HOSTING_EBPF_SHA256 / interfaces env, or socket not created.
8. UID/GID V3 enabled in example YAML (enable_uid_gid_filtering: true) despite incomplete ABI.
9. OS/kernel "fully supported" table from internal ops copied as a guarantee (not certified).

Diagnostic commands/endpoints #

Units (Enterprise tree):

sudo systemctl status zwall-agent
sudo journalctl -u zwall-agent -n 200 --no-pager
sudo systemctl status zwall-hosting-ebpf-loader
ls -l /run/zwall-hosting-ebpf/observer.sock
uname -rCopy

GET /api/v1/dashboard/ebpf-health
GET /api/v1/dashboard/ebpf-health?agent_id=<id>Copy

Enterprise only. Community router: no ebpf-health handler.

On-node maps (Virtualization): zwallctl map ls (default pin dir /sys/fs/bpf/fw).

Example Agent keys (configs/agent.example.yml):

• ebpf.program_path: /opt/zcs/ebpf/firewall.o
• ebpf.tc_program_path: /opt/zcs/ebpf/firewall_egress.o
• ebpf.xdp_attach_mode: auto
• ebpf.min_kernel_version: "5.8.0"
• ebpf.strict_kernel_version_check: true
• ebpf.verify_artifact_integrity: true
• ebpf.allow_unsigned_artifacts: false
• ebpf.checksums_manifest_path: /opt/zcs/ebpf/ebpf_checksums.txt

Hosting examples set verify_artifact_integrity: false on that profile’s YAML — Hosting still requires the loader SHA256 env (ZWALL_HOSTING_EBPF_SHA256). Do not copy Hosting’s integrity flag onto a Virtualization Agent.

What each result means #

Hosting loader: root owns verified load/attach, then drops to CAP_SYS_ADMIN (map reads) and CAP_NET_ADMIN (owned detach). The unprivileged Agent gets no capability and only the category-safe UNIX socket.

Safe remediation #

1. Virtualization: run zwall-agent.service as shipped (root + bounded caps). Do not empty CapabilityBoundingSet and expect attach to work.
2. Ensure /sys/fs/bpf is mounted and in the unit ReadWritePaths.
3. Use objects from the admitted release; keep verify_artifact_integrity: true on Virtualization.
4. If attach fails: try generic only as a controlled fallback. Do not disable integrity.
5. Hosting: start loader with --object /opt/zwall-hosting/ebpf/hosting_observe.o and the SHA256 env; confirm socket; then start the Agent.
6. Set enable_uid_gid_filtering: false until engineering publishes a complete contract.
7. Do not attach firewall programs on Hosting nodes.

Do-not-do warnings #

• Do not document UID/GID V3 as supported even though example YAML enables it.
• Do not publish a “fully supported” distro/kernel table.
• Do not publish PPS or latency.
• Do not set allow_unsigned_artifacts: true in production.
• Do not grant Hosting Agent CAP_NET_ADMIN / CAP_SYS_ADMIN.
• Do not copy Hosting verify_artifact_integrity: false onto Virtualization.
• Do not treat Community README 4.15+ as the Enterprise Virtualization example 5.8.0 check.

Escalation condition #

Escalate when:

• kernel meets the profile’s configured min_kernel_version, capabilities match the unit, integrity verifies, and attach still fails on both native and generic
• Hosting pass-only object drops traffic
• owner needs a single OS matrix — documentation cannot close that

Related documentation #

• XDP/TC
• Virtualization troubleshooting
• Systemd units
• Least privilege
• Enforcement
• System requirements

Edition #

SHARED Virtualization eBPF. Hosting loader and observer socket ENTERPRISE_ONLY. ebpf-health ENTERPRISE_ONLY.

Status #

SUPPORTED for unit/config diagnosis. The kernel-matrix and UID/GID-contract issues remain product gaps.