How to get an alert when a cron job or timer silently stops running (Uptime Kuma push monitors)

Updated:  /  Home Lab

My NAS runs a bunch of scheduled jobs: a content pipeline three times a day, a nightly backup, a nightly docs refresh and a weekly price check. Most of them already send me a Telegram message when something goes wrong. But that only works if the job actually runs. When a job never starts, nothing sends anything, and silence looks exactly like “all good”.

That’s exactly what happened to me. My NAS rebooted overnight without me knowing, and on UGOS a reboot can leave every systemd user timer dead (I wrote about that boot trap in my UGREEN cron jobs guide). No job ran, so no job complained.

In this guide, I will show you how I use Uptime Kuma push monitors as a “dead man’s switch”: every job pings Kuma when it finishes, and Kuma alerts me when a ping doesn’t arrive in time.

How it works

A normal Uptime Kuma monitor checks something every few minutes: it opens a page, pings an IP, asks a DNS server. A push monitor works the other way around. Kuma gives you a URL and waits. Your job calls that URL when it’s done, and if Kuma doesn’t hear from it within the interval you set, the monitor goes down and you get an alert.

This catches the failures a script can’t report itself:

  • the timer or cron job never fired,
  • the job hung and never finished,
  • the whole machine was off or rebooting.

There’s one more trick I use a lot: the job can also push a down status with a message. So a job that runs and fails can tell Kuma why, and it lands in the same Telegram chat as everything else.

Here’s what we need to get started:

  • Uptime Kuma, running somewhere your jobs can reach. Mine runs in Docker on the NAS, at uptime.lan.
  • A notification set up in Kuma. Mine is Telegram, through the bot of my Hermes agent, so all my alerts land in one chat.
  • curl (or wget) on the machine that runs the jobs.

1. Create a push monitor

In Uptime Kuma, click Add New Monitor and set:

  • Monitor Type: Push
  • Friendly Name: the job’s name and how often it runs. I name mine like nas-backup (nightly) or pro-review stock sweep (weekly), so the alert tells me what to expect.
  • Heartbeat Interval: how long Kuma waits for a ping before it marks the job as down.
  • Heartbeat Retry Interval: how often Kuma checks again once it’s down. I use 3600 (one hour).
  • Notifications: tick your notification.

For the interval, don’t use the exact schedule of the job. A daily job doesn’t always finish at the same minute, so give it some slack. I use 93600 seconds (26 hours) for every daily job, and 691200 seconds (8 days) for my weekly one. That way, one late run doesn’t wake me up, but a missed day does.

Don’t skip the notification checkbox. In Kuma 1.x, a new monitor doesn’t get your notification automatically unless you made it the default. In October, I found out that 15 of my monitors had no notification attached at all. They turned red in the dashboard, and nobody told me.

Save the monitor and copy the Push URL. It looks like this:

http://uptime.lan/api/push/AbCdEf1234?status=up&msg=OK&ping=

The part after /api/push/ is a token. Anyone who has it can mark your job as up, so treat it like a password. I keep each URL in the job’s own .env file, not in the script.

2. Ping it at the end of the job

All my shell jobs share a small helper in a lib.sh file. Copy & paste the following function:

# heartbeat "<push url>" [message] - Uptime Kuma push monitor ping.
# No-op when the URL is unset; never fails the caller.
heartbeat() {
  [ -n "${1:-}" ] || return 0
  curl -fsS -m 10 -o /dev/null --get --data-urlencode "status=up" \
    --data-urlencode "msg=${2:-OK}" "${1%%\?*}" \
    || echo "WARN: heartbeat ping failed (non-fatal)" >&2
}

A few things I did on purpose here:

  • ${1%%\?*} removes the query string from the URL. Kuma’s copy button gives you the URL with ?status=up&msg=OK&ping= already on it, so I can paste it as is and the function adds its own parameters.
  • -m 10 gives up after 10 seconds. A slow Kuma should never hold up the job.
  • If the ping fails, it prints a warning but doesn’t fail the job. The monitor going down is the alert in that case.
  • If the URL isn’t set, it does nothing, so I can run the same script on my laptop without a monitor.

Then call it as the last line of the job, only after the real work succeeded:

source scripts/lib.sh

# ... the actual work ...

heartbeat "${KUMA_PUSH_PROREVIEW_PIPELINE:-}" "$RUN_MODE $SLUG ${TOTAL}s"

The message is optional, but I recommend it. Kuma shows it next to every heartbeat, so the monitor’s history doubles as a log: in my case, which mode the pipeline ran in, which article it worked on and how many seconds it took.

You could also ping Kuma from the systemd unit itself (with ExecStartPost=), without touching the script. I prefer pinging from the script, because it decides what counts as a success, and it can say what it did.

If your jobs run in a small Alpine container (like the cron container from my UGREEN guide), there’s no curl by default, but BusyBox wget works just as well:

wget -q -O /dev/null "http://uptime.lan/api/push/AbCdEf1234?status=up&msg=OK"

3. Report failures as “down”, not just silence

A job that runs and fails could simply skip the ping, and Kuma would alert me a day later. But I don’t want to wait 26 hours to learn that my backup failed at 03:30.

So my Python jobs push status=down with the reason right away. This is the function from my nightly docs job (slightly shortened):

import urllib.parse, urllib.request

def heartbeat(url, up, msg):
    if not url:
        return
    q = urllib.parse.urlencode({"status": "up" if up else "down", "msg": msg[:200], "ping": ""})
    try:
        urllib.request.urlopen(f"{url}{'&' if '?' in url else '?'}{q}", timeout=20).close()
    except Exception as e:
        print(f"warn: kuma push failed: {e}")

Kuma marks the monitor down at once, and the alert includes the message. These are two real ones from my nightly backup:

failed: <urlopen error [Errno -3] Temporary failure in name resolution>
failed: <urlopen error EOF occurred in violation of protocol (_ssl.c:2393)>

Keep the message short (I cut it at 200 characters). It’s an alert, not a log file.

4. Test it

Run the job once by hand and open the monitor in Kuma. You should see a green heartbeat with your message. If it stays grey, check that the machine running the job can reach Kuma at all (curl -I http://uptime.lan is a quick test).

Next, test the case that matters: the one where nothing happens. You don’t have to wait 26 hours for that. Temporarily set the Heartbeat Interval to something short, like 60 seconds, don’t run the job, and wait for the alert in your notification channel. Then set the interval back.

What it caught so far

I set up my first push monitors at the end of September, and now I have seven of them. A few alerts that made me glad I did:

  • The reboot I didn’t know about. On the morning of 11 October, four monitors went down within seconds of each other: the pipeline, its health check, the nightly backup and the docs job. Four different jobs failing at the same time pointed straight at the NAS: it had rebooted the day before without me noticing, and all my timers were dead. Kuma’s alert was how I found out, the same morning.
  • A backup that failed, loudly. Twice the nightly backup pushed down with the error message above, instead of failing quietly in a log I’d never read.
  • The docs job reporting something new. My nightly docs job pushes down when it finds a new problem (for example, a monitor pointing at a port where nothing listens anymore). That’s a job working, but still something I want to know about the same morning.

That’s it! You now get an alert when a scheduled job runs and fails, and also when it doesn’t run at all.

Limitations

  • Nothing watches Kuma. My Kuma runs on the same NAS as the jobs. If the NAS is off, Kuma is off too, and I get no alert at all. A monitor that lives somewhere else (a second Kuma on another machine, or a hosted service like Healthchecks.io) would fix that.
  • One late run can still alert. If a daily job is slower than usual and finishes after the 26-hour window, Kuma alerts, and then it recovers on the next ping. That’s fine by me, I’d rather get one false alarm than miss a dead job.
  • I’m still on Uptime Kuma 1.x. Version 2 changes the database layout, so that upgrade needs some planning. Push monitors exist in version 2 too.

If you use something else as a dead man’s switch for your home lab, let me know.