$ systemctl --user list-timers # what I run in production

Not a tutorial. A Raspberry Pi 5 I operate as a small production environment: 229 scheduled jobs and 400+ systemd services that source leads, apply to jobs, publish sites, take payments, and watch each other. Zero inbound ports: every public endpoint rides an outbound-only Cloudflare Tunnel. When a unit fails, it pages my phone; a self-heal loop restarts it with rate limits and a watchdog checks the healers.

$ fleet-status # measured on the box

systemwhat it doesmeasured
systemd user manager
229 timers, 402 service units; oneshot + timer, never cron
229 / 402
failure paging
OnFailure=alert@%n on every unit → phone, exact-repeat suppressed
every unit
self-heal + watchdog
10-min healer with restart limits; daily watchdog checks the healers
10 min / 24 h
edge
outbound-only Cloudflare Tunnels; origins on loopback; zero inbound ports
3 tunnels
DNS
Pi-hole → Unbound recursive resolver with DNSSEC for the LAN
:53 → :5335
operator access
SSH over Tailscale only; WireGuard site-to-site to the AWS lab VPC
3 nodes
backups
nightly, encrypted, two destinations, restore drill
nightly
read from systemctl --user list-timers --all, systemctl is-active and tailscale status on 2026-10-01 · patterns and postmortems: github.com/ravikus1457/fleet-ops

$ cat practices.md

⚙ Fleet operations (SRE habits)

  • systemd units + timers
  • OnFailure → Telegram paging
  • Self-heal with backoff
  • Watchdog of the watchdogs
  • Persistent journald
  • Nightly backups, verified restores
  • Memory guards (earlyoom, cgroup caps)
  • Runbooks for every unit

⇄ Network edge

  • Cloudflare Tunnel ×3 (no open ports)
  • Cloudflare Pages
  • Pi-hole + Unbound (DNSSEC) LAN DNS
  • Tailscale mesh (Pi · laptop · phone)
  • WireGuard site-to-site
  • nftables default-deny
  • LAN watch: new-device alerts

✓ How I know it works

Every detector is fault-tested: inject the exact failure it exists to catch, watch it go red, restore. Monitors read the thing itself (a heartbeat, a version stamp), never a proxy for it. A check that has never failed is a check that has never been tested.

  • Fault injection
  • Heartbeats over proxies
  • Postmortems

$ cat architecture/edge.md

How the fleet reaches the internet without exposing it: services bind to loopback on the Pi, cloudflared opens outbound tunnels to Cloudflare, and Cloudflare terminates TLS and routes each hostname to its tunnel. Operators reach the box over Tailscale; the LAN resolves through Pi-hole → Unbound with DNSSEC. The AWS labs attach to the same Pi over a WireGuard site-to-site tunnel.

A Raspberry Pi 5 on a dark desk with amber and blue light beams rising from it to a glowing wireframe cloud
Pi fleet: loopback-only services, systemd, nftables, Pi-hole + Unbound cloudflared outbound tunnels · WireGuard to AWS · Tailscale for the operator Cloudflare edge + AWS VPC: TLS at the edge, security groups, least privilege

$ ls products/ # shipped, taking real traffic

$ PaperReel — a paid product

Video abstracts for research papers. Stripe Checkout → signed webhook → systemd pipeline (extract → script → render) → signed download links. Landing on Cloudflare Pages, API through a Tunnel, hard per-service spend caps, fault tests that can fail.

  • Stripe webhooks
  • Cloudflare Pages + Tunnel
  • Python
  • ffmpeg
  • Cost caps in code
paperreel.pages.dev ↗

rsvp Wedding platform — shipped

Full-stack event site for a four-day wedding, hundreds of guests: RSVP with email fan-out, QR guest-camera uploads, private album with on-demand renditions, address revealed only after RSVP. Served from the Pi through a Tunnel; loopback-only origin, per-IP rate limits, EXIF scrubbing.

  • Python HTTP service
  • Cloudflare Tunnel
  • HEIC→JPEG
  • Byte-range streaming
raviandaachal.com ↗

next: the AWS labs →