Silent When Healthy

Once you run more than a handful of scheduled jobs, the dangerous failure is not the one that throws an error. It is the one that produces nothing at all.

I run 32 scheduled jobs across my work and personal projects: status syncs into Slack, a finance sweep, inbox readers, nightly CI, Vercel crons, a weekly growth review. Most of them were written in an afternoon with Claude Code. That speed is the point, and it is also the problem. Building a job now costs almost nothing. Noticing that it has stopped costs exactly as much as it always did.

Three ways a job fails

1. It ran, failed, and said so into the void. A sync logged msg_too_long on four consecutive hourly runs. The log was correct. Nobody read it, and a status page stayed ten hours stale.
2. It stopped running. Five workflows were switched off by hand. No log anywhere mentioned it, because nothing ran. Error monitoring cannot see this failure, since no errors looks exactly like success.
3. It ran, reported success, and did the wrong thing. A sync whose source folder had moved found nothing to do, exited cleanly, and logged no failure.

Only the first leaves an error. So every check asks "when did this last finish?" before it asks "did it fail?"

The principle

Keep one registry of everything that is supposed to run, and one checker over it. The checker says nothing when all is well.

That silence is deliberate. A morning report that lists 32 green ticks trains you to skim it, and a surface you skim is no longer a surface. When the checker prints nothing, nothing needs you. When it prints something, it is worth reading.

Four rules that make it work

Every job writes a trailer. The runner logs a start line, runs the work, and logs a done line only after the work exits. A run that started and died must look different from a run that finished, and only a trailer can show that.

Every entry carries a fix. The registry refuses a job that does not say what a human should do when it breaks, written as a command they can paste. If you cannot write the fix, the alert will fire and be ignored, and an alert that is ignored teaches you to ignore the next one.

Assert the result, not just the exit code. For the third failure mode, each job can declare a line that must appear in a healthy run (uploaded <file>, synced <x> -> <y>). Test that pattern against the job's entire log history before trusting it. A pattern that fires on a legitimate quiet day does more harm than no pattern.

Watch the watcher. The checker includes itself. When a log moves or a CLI loses its auth, a real check quietly becomes "unknown", and a partly blind checker looks exactly like a working one. So the checker fails loudly when it cannot see.

What laptops taught me

Two macOS behaviors produced most of the false alarms.

A scheduled job only fires while the machine is awake, so a healthy hourly job reads "eight hours late" every morning. The checker now measures lateness in awake time.

Worse, macOS often runs a calendar job inside a maintenance wake that lasts 10 to 45 seconds and then sleeps again with the job still running. The break surfaces as whatever was in flight: a DNS failure, an expired token, a broken certificate store, a slow git remote. Four different wrong causes for one real one. I spent two days chasing an OAuth token that was valid the whole time. The fix is to hold the machine awake for exactly as long as the job runs, and to register the scheduled wake itself as a job, because an OS update can clear it without a word.

The general lesson: register the things your jobs silently depend on, not only the jobs.

Try it

The checker is open source: jobwatch. It is one Python file with no dependencies, an example registry, the runner convention, and a test suite that needs none of your real jobs. Copy the example, list what you run, and see which of your jobs have already stopped.

It is read-only by design. It never restarts or repairs anything, because a repair loop that misfires at 3am is worse than the failure it was meant to fix.