Watch
dog

Probes run checks against your devices. The core stores the results, evaluates state and sends notifications. That is the whole system — and the discipline was in refusing to let it become more than that.

Commercial network monitoring tends to arrive as an appliance: a database cluster, a message broker, a metrics store, an agent fleet, and a licensing server to hold it together. For the estates we actually see — a few hundred devices, one or two sites, an admin who is not full-time on monitoring — that is the wrong shape. Watchdog is the same capability set delivered as a standalone native service with PostgreSQL as its only always-on dependency.

The split that matters is core and probe. The core owns state and truth; the probe owns execution. They speak over a versioned wire protocol that lives in a shared package, which is what makes a remote probe on another network exactly the same code as the local one.

Because the probe is a separate process with a defined protocol, three things fall out for free: a probe can sit behind NAT on a client site and dial home, a probe crash cannot take the API down, and the scheduler can be reasoned about — and load-tested — without a database in the loop.

Roughly 43 sensor types ship in the catalogue, and the number is allowed to grow because adding one is a bounded piece of work: a single file implementing six members of the SensorType contract. No registry edit, no migration, no touching the scheduler.

The public sensor reference is generated from the catalogue rather than written alongside it, so the documentation cannot drift from what the system actually supports. The API reference is generated from the OpenAPI document the same way.

A monitoring system is judged on the alerts it doesn't send. The hard part is not detecting that a sensor went down — it is deciding whether anyone should hear about it, given that the device is in a maintenance window, its parent is already down, and someone paused the whole group on Friday.

So sensor state and pause precedence are specified as a state machine with written transitions and invariants, in its own document, before the code. Notification routing sits downstream of that machine rather than inside it.

Production is a standalone native service — systemd on Linux, WinSW on Windows. That is a deliberate reversal of our usual Docker-first default, and it is the right call here: the people installing a monitoring system are frequently the people who do not want a container runtime added to the box they are monitoring from.

Sizing guidance comes from measurements rather than estimates, and both the backup-restore path and the release-candidate field test have written runbooks — including restoring onto a host that is not the one the backup came from, which is the case that actually matters.

The documentation set is the part of this project we would point at first. It is indexed by a README that says which document answers which question, and it is structured so that a reader arriving with a symptom, a scoping question or a change to make each has a different front door.

Phases 0 through 9 are complete: the object model, the scheduler, the state machine, notifications, the web UI, the sensor catalogue, discovery, remote probes and RBAC are all in. Phase 10 — hardening and release — is in progress.

What remains is release work rather than feature work. The Ubuntu half of the field test is proved and running in CI; the rest of it is waiting on hardware.